Cloud storage, scaling and load balancing notes — Unit 3

Data storage in the cloud is fundamentally different from installing a hard drive in a local computer. The cloud abstracts the physical media and presents storage as a scalable service accessible over a network.

⇒1.1 The Need for Distinct Types

A database requires ultra-low latency reads/writes. A backup archive requires massive capacity at the lowest possible cost, but speed is irrelevant. Because of these conflicting requirements, cloud providers offer three primary types of storage: Block, File, and Object.

Block storage breaks data up into chunks (blocks) and stores them as separate pieces, each with a unique identifier. The Storage Area Network (SAN) places these blocks wherever it is most efficient.

⇒2.1 Characteristics

• Access Method: Accessed via a server OS, which formats the block storage with a file system (like NTFS or ext4). It behaves exactly like a physical hard drive plugged into a server.
• Performance: Extremely fast, low-latency I/O operations.
• Use Case: Operating system boot drives, transactional databases (SQL), and applications requiring high-speed read/write access.
• Examples: AWS EBS (Elastic Block Store), Azure Disk Storage.

File storage organises data in a hierarchical structure of files and folders (directories). It is the same paradigm used on personal computers.

⇒3.1 Characteristics

• Access Method: Accessed over a network using protocols like NFS (Network File System) or SMB (Server Message Block).
• Shared Access: Multiple Virtual Machines or servers can attach to the same file storage simultaneously and read/write the same files.
• Use Case: Corporate file shares, massive content repositories for web servers, and big data analytics.
• Examples: AWS EFS (Elastic File System), Azure Files.

Object storage ditches the file hierarchy entirely. It stores data as 'objects' in a massive, flat data lake. Each object consists of the data itself, extensive metadata, and a globally unique identifier.

⇒4.1 Characteristics

• Access Method: Accessed entirely via HTTP REST APIs (GET, PUT, DELETE). You cannot format it with an OS or install a database on it.
• Scalability: Infinite. You can store exabytes of data without worrying about 'running out of space' or managing disk volumes.
• Use Case: Storing photos for a social network, streaming video files (Netflix), backups, and archiving.
• Examples: AWS S3 (Simple Storage Service), Google Cloud Storage.

Cloud providers offer databases as managed PaaS offerings, removing the burden of installing database software, patching it, and configuring replication.

⇒5.1 Relational Databases (SQL)

Used for structured data with strict schemas (e.g., financial transactions). Managed services like Amazon RDS or Azure SQL handle automated backups, failover to secondary zones, and software patching. You simply connect your application via a connection string.

⇒6.1 The Need for NoSQL

Relational databases scale vertically (requiring a bigger, more expensive server). Web-scale applications require databases that can scale horizontally (spreading data across hundreds of cheap servers).

• Key-Value Stores: Highly optimized for massive read/write speeds. E.g., Amazon DynamoDB, Azure Cosmos DB.
• Document Stores: Store semi-structured data like JSON. Excellent for rapid development. E.g., MongoDB Atlas.
• In-Memory Caches: Store frequently accessed data in RAM for microsecond latency. E.g., Amazon ElastiCache (Redis).

Scalability is the measure of a system's ability to increase or decrease in performance and cost in response to changes in application and system processing demands.

⇒7.1 Vertical Scaling (Scaling Up)

Increasing the capacity of a single machine by adding more RAM, CPUs, or replacing it with a larger server. It is easy to implement (no code changes needed) but has a hard physical limit. If your server is already the largest available, you cannot scale up further.

⇒8.1 The Paradigm Shift

Instead of making one server bigger, you add more servers to a pool of resources. If traffic doubles, you double the number of web servers. This offers theoretically infinite scalability.

However, it requires the application architecture to be designed for it. The application must be stateless. If User A logs into Server 1, and their next request hits Server 2, Server 2 must know they are logged in. State must be externalized to a database or cache.

The cloud allows infrastructure to scale automatically based on rules, without human intervention.

⇒9.1 How it Works

• Metrics: The system continuously monitors CPU utilization, network traffic, or memory usage across the server fleet.
• Thresholds: You define a rule: 'If average CPU across all instances exceeds 70% for 5 minutes, add 2 more servers.'
• Action: The cloud provider automatically provisions the VMs, boots the OS, installs your code, and registers the new servers to start receiving traffic.

Crucially, auto-scaling also scales in when traffic drops, saving money.

If you have 10 identical web servers (scaled horizontally), how does the user's browser know which one to connect to? It doesn't. It connects to a Load Balancer.

⇒10.1 The Traffic Cop

A load balancer is a device (usually a managed cloud service) that acts as a reverse proxy, distributing network or application traffic across a number of servers. It ensures no single server bears too much demand.

• Round Robin: Sends requests sequentially to each server.
• Least Connections: Sends the request to the server with the fewest active connections.

⇒11.1 Layer 4 (Network) Load Balancing

Operates at the transport layer, routing traffic based purely on IP addresses and TCP/UDP ports. It is extremely fast and can handle millions of requests per second because it does not inspect the contents of the packets.

⇒11.2 Layer 7 (Application) Load Balancing

Operates at the application layer. It inspects the actual HTTP/HTTPS traffic. This allows it to make complex routing decisions. For example, routing requests for `/images/` to Server Group A, and requests for `/api/` to Server Group B.

A load balancer only provides reliability if it knows which servers are healthy.

⇒12.1 The Mechanism

The load balancer continuously sends 'health check' requests (e.g., pinging a `/health` HTTP endpoint) to all registered servers.

If a server fails to respond (or returns a 500 Error) a set number of times, the load balancer marks it as 'unhealthy' and instantly stops sending user traffic to it. The Auto Scaling group will then terminate the broken server and launch a fresh replacement.

What happens if an entire geographic region (data center) goes offline? Regional load balancing isn't enough.

⇒13.1 DNS-Based Routing

Cloud providers offer managed DNS services (like AWS Route 53) that act as global traffic cops. When a user in Tokyo types `google.com`, the DNS service detects their location and routes them to the Tokyo data center's load balancer to minimize latency.

If the Tokyo data center fails health checks, the DNS service automatically updates its records to route all Asian traffic to a backup data center in Singapore, ensuring global fault tolerance.

⇒14.1 Content Delivery Networks (CDNs)

To reduce global latency, CDNs cache static assets (images, videos) at Edge Locations near the users. This offloads massive traffic from the central servers.

⇒14.2 Statelessness

To utilize Auto Scaling, web servers must be stateless. User sessions and uploaded files must be saved in external systems like Redis or S3, so any server can handle any request and servers can be destroyed without data loss.