Distributed system architectures and communication notes — Unit 1

A distributed system is a collection of independent computers that appears to its users as a single coherent system. The nodes in the system coordinate their actions by passing messages.

⇒1.1 Key Characteristics

• Concurrency: Components execute in concurrent processes. There is no global clock or synchronization by default.
• No Global Clock: Programs coordinate by exchanging messages, but cannot rely on exact time synchronization across nodes.
• Independent Failures: Some components can fail while others keep running. The system must tolerate partial failures.

Building a distributed system is complex. A successful design aims for several key objectives.

⇒2.1 Transparency

The main goal is to hide the fact that its processes and resources are physically distributed across multiple computers. A truly transparent system hides:

• Access: Differences in data representation and how resources are accessed.
• Location: Where a resource is located.
• Migration: That a resource may move to another location.
• Failure: The failure and recovery of a resource.

Scalability implies the system can handle growth without degrading performance.

⇒3.1 Dimensions of Scalability

• Size Scalability: Can easily add more users and resources to the system.
• Geographical Scalability: Users and resources may lie far apart.
• Administrative Scalability: Can easily be managed even if it spans many independent administrative organizations.

Scaling techniques include hiding communication latencies (asynchronous communication), distribution (splitting data across servers), and replication (copying data for faster local access).

An architectural style is formulated in terms of components, the way that components are connected, the data exchanged, and how they interact.

⇒4.1 Common Styles

• Layered Architecture: Components are organized in layers. Layer N only calls Layer N-1.
• Object-Based Architecture: Each object corresponds to a component, connected via an RPC mechanism (e.g., CORBA).
• Data-Centered Architecture: Processes communicate by modifying a shared repository (e.g., a shared distributed database or web-based services).
• Event-Based Architecture: Processes communicate via the propagation of events. Allows loose coupling.

⇒5.1 Centralized Architectures (Client-Server)

The most common paradigm. A server provides a specific service, and clients request it. It has clear separation of duties but the server is a single point of failure and a bottleneck.

⇒5.2 Decentralized Architectures (Peer-to-Peer)

All nodes are equal and act as both clients and servers (Servents). They are highly resilient and scalable. Examples include BitTorrent and early Skype.

P2P networks are categorized by how they organize nodes and route requests.

⇒6.1 Structured P2P

The overlay network is constructed using a deterministic topology (like a ring or tree). Data is placed at specific locations (often using Distributed Hash Tables - DHTs). Finding data is efficient (O(log N)). Example: Chord.

⇒6.2 Unstructured P2P

Nodes form random connections. Finding data requires flooding the network with queries. It is robust to high node churn (nodes joining/leaving constantly) but finding rare data is slow. Example: Gnutella.

Because there is no shared memory, communication relies entirely on the network.

⇒7.1 Network Protocols

Most distributed systems are built on top of TCP/IP. The lower OSI layers handle the raw transmission, while the higher layers (Transport/Application) handle the logic of the distributed system itself.

However, programming raw TCP sockets is tedious. Developers need higher-level abstractions like RPC, RMI, or Message-Oriented Middleware (MOM).

RPC aims to make a remote network call look exactly like a local function call in the code.

⇒8.1 The Mechanism

• 1. The client calls a local 'client stub' function.
• 2. The stub packages the parameters into a message (marshaling) and makes a system call to send it.
• 3. The network transmits the message to the server.
• 4. The server OS hands the message to a 'server stub', which unmarshals the parameters.
• 5. The server stub calls the actual server function.
• 6. The result is marshaled and sent back following the same steps in reverse.

⇒9.1 Parameter Passing

Passing 'Call-by-Value' is easy (just copy the integer). Passing 'Call-by-Reference' is practically impossible because the server and client have different memory spaces. Pointers are meaningless across a network.

⇒9.2 The Fallacies of Distributed Computing

RPC tries to hide the network, but developers must remember the network exists. The fallacies include: 'The network is reliable', 'Latency is zero', 'Bandwidth is infinite', 'Topology doesn't change'. Assuming these are true leads to brittle systems.

RPC is inherently synchronous (blocking). If the server is down, the client hangs. Message-oriented communication provides asynchronous solutions.

⇒10.1 Transient vs Persistent

• Transient Communication: The message is stored only as long as both the sender and receiver are executing. If the receiver is offline, the message is dropped (e.g., raw sockets).
• Persistent Communication: A message is stored at a communication server as long as it takes to deliver it to the receiver. The sender and receiver don't need to be online at the same time.

MOM provides persistent, asynchronous communication via Message Queues.

⇒11.1 Message Queuing Systems

Applications communicate by inserting messages in specific queues. A receiver pulls messages from its queue. The middleware guarantees the message will stay in the queue until it is read or expires.

This enables massive scalability. If a web server receives a heavy spike of requests, it doesn't process them immediately. It simply drops them into a RabbitMQ or Kafka queue. Worker servers pull from the queue at their own pace, preventing system crashes.

RPC and MOM deal with discrete messages. Stream-oriented communication deals with continuous media (audio, video).

⇒12.1 Timing is Everything

In discrete communication, arriving a few milliseconds late is fine as long as the data is accurate. In continuous media, timing is critical. If audio packets arrive out of order or with highly variable delay (jitter), the sound is distorted.

Streaming protocols (like RTP) often prioritize timely delivery over guaranteed delivery (using UDP instead of TCP). It's better to drop a frame of video than pause the stream for 2 seconds to recover it.

Sending data to multiple receivers.

⇒13.1 Approaches

• Hardware Multicast: The network switch copies the packet to all ports. Highly efficient but limited to local networks.
• Application-Level Multicast: Nodes organize into an overlay network (like a tree). The sender sends the message to the root node, which copies and forwards it to its children, and so on. Used for Internet-wide broadcasting.

• Distributed Systems: Independent computers appearing as a single system. Goals: Transparency, Scalability, Fault Tolerance.
• Architectures: Client-Server (centralized) vs Peer-to-Peer (decentralized).
• Communication: RPC hides the network (synchronous). Message Queues (MOM) provide persistent, asynchronous decoupling.
• Streams: Continuous media requires timing guarantees (QoS) over raw data reliability.