A grid is a group of connected computers that share their power to work as one, eventhough some resources are restricted to some users.
Grid computing is considered a massive team computers working without considering if one computer is fast or slower, the goal is to use their analysis time to get their job done.
Three ways this work is done:
Remote Execution: Running a standard application on a grid machine instead of a local one.
Parallel Processing: Splitting a single application into sub-tasks that run simultaneously across multiple processors.
High-Throughput Computing: Running multiple instances of the same application across various nodes.
Beyond processing power, grids integrate the storage capacity of all connected machines to create one shared storage space, high-performance pool. This storage is categorized into two main types:
Volatile Memory (Quick and Temporary): Attached directly to processors, offering high-speed access for temporary caching.
Secondary Storage (Slow and Permanent): Hard disks and permanent media managed by networked file systems to ensure reliability and security.
Key Functional Advantages
Grid-based storage provides several benefits over traditional local storage:
Unified Capacity and Namespace: By spanning files and databases across multiple machines, grids eliminate size restrictions and allow users to access data through a single, consistent name, regardless of its physical location.
Database Federation: Specialized software can put together separate databases into one database system and function as one.
Data Striping: This technique distributes data across multiple drives at once to achieve transfer speeds faster than any single disk could provide essential for high-bandwidth tasks.
Reliability & Performance: Features like journaling protect against data loss during failures, while advanced synchronization reduces conflicts when multiple users update shared data simultaneously.
Modern grid computing is made practical by the rapid growth in communication capacity, which has evolved due to limited bandwidth of early distributed systems. This communication is categorized into two main types:
Internal Communications: Essential for distributing jobs and data to various nodes within the Grid. Since data is often on a different machine than the processor, internal bandwidth frequently acts as the primary bottleneck for grid utilization.
External Communications: Grid nodes often have independent connections to the internet. When these paths are separate, they aggregate to increase total available bandwidth, which is vital for data-heavy tasks like building search engines.
Beyond hardware, grids efficiently manage software licenses as a shared resource, which can lead to significant cost savings for organizations. Rather than purchasing expensive software for every node, the grid uses two primary strategies:
Targeted Execution: Jobs requiring specific, high-cost software are automatically routed only to the few machines where that software is installed.
Concurrent License Management: For software licensed by simultaneous use rather than per machine, specialized management tools track active copies in real-time. This ensures the grid never exceeds its legal limit while allowing the software to be available across all nodes.
Special equipment, capacities, architectures, and policies
Because grid environments are heterogeneous systems, they must account for the diverse architectures, operating systems, and hardware capacities of their member machines. This diversity requires job assignment criteria to ensure compatibility and policy compliance:
Technical Compatibility: Schedulers must match software requirements to the specific hardware and operating system of a node to ensure applications can actually execute.
Artificial Resource Attributes: Administrators can create custom tags or labels for machines to enforce specific usage policies. These go beyond technical specs to include:
Research Focus: Dedicating specific nodes exclusively to fields like medical research.
Ethical or Policy Constraints: Restricting certain machines from participating in specific types of work, such as military projects.
Governance and Classification
To make these artificial resources effective, the grid requires a certification procedure. This ensures every job is properly classified before the scheduler matches it to a node that meets both the technical and ethical requirements of the task.
In a grid environment, work is organized into a hierarchy where a high-level application is broken down into smaller, executable units called jobs, subjobs, or work units. These programs are dispatched across the grid to perform diverse tasks, from complex data processing to hardware control.
Execution and Workflow
While many grid applications are designed for parallel execution to save time, their efficiency is often dictated by a structured workflow:
Dependencies: Jobs often cannot run simultaneously; some may require specific input data to be moved first, while others must wait for the output of a prerequisite job.
Dynamic Hierarchy: As jobs process data, they can dynamically spawn additional subjobs, creating a complex execution tree.
Aggregation: The final stage of any grid application involves collecting and assembling the results from these scattered tasks into a single, cohesive output.
Scheduling, reservation, and scavenging
The grid’s efficiency relies on how it matches work to machines. This process ranges from manual selection to automated systems that manage how and when resources are used.
Methods of Job Assignment
Manual vs. Automated: While users can manually pick a machine, advanced grids use schedulers (or resource brokers) to automatically find the best fit based on real-time availability.
Scavenging: This unobtrusive mode uses idle cycles on machines. If a local user performs a task, the grid job is downgraded to the lowest priority or suspended. This maximizes efficiency but makes completion times unpredictable.
Dedicated Resources and Reservation: For predictable performance, machines can be dedicated solely to the grid. Reservations allow users to book resources in advance to meet strict deadlines.
While grid computing is driven by software, the level of automation varies significantly. In early or resource limited grids, many functions are handled through manual procedures. However, as environments mature, these tasks shift toward autonomic processing where sophisticated software manages the grid independently.
When designing or deploying a grid, it is crucial to view these software functions as a spectrum of capabilities. Even if a grid begins with manual oversight, the goal is to implement robust software layers over time to handle the increasing complexity of the environment.
Grid systems exist on a spectrum of scale and complexity, evolving from simple local clusters to global collaborative networks. This progression is categorized into two main types:
Intragrid: Internal Organizational Scaling
An intragrid operates within a single organization, typically following this growth path:
Homogeneous Clusters: The simplest form, consisting of identical machines on a local network. These are easy to manage but limited in scope.
Heterogeneous Integration: As the grid expands across departments, it incorporates diverse architectures and requires automated scheduling and policy management.
Geographic Expansion: Intragrids can span different cities using dedicated connections or VPNs. At this stage, security and hierarchical control become vital to prevent bottlenecks and protect departmental data.
Intergrid: Global & Collaborative Scaling
An intergrid occurs when the network crosses organizational boundaries to facilitate wide-scale collaboration.
Resource Brokering: These grids allow for the trading or purchasing of computing power as a utility from trusted suppliers.
Maximum Security: Because they involve multiple independent entities, intergrids require the highest level of security and rigorous certification to manage shared interests and sensitive data.