Edge computing, cloud integration and data handling notes — Unit 4

In the first generation of IoT architectures, every sensor simply forwarded raw data directly to a centralized Cloud server (like AWS or Azure). The cloud performed all computations and sent commands back. As the number of devices exploded, this 'Cloud-only' model began to break down.

⇒1.1 The Three Constraints

• Latency: If an autonomous car relies on a cloud server 1000 miles away to decide whether to brake for a pedestrian, the 200-millisecond network round-trip time is fatal.
• Bandwidth: A single factory floor with hundreds of high-definition vibration analysis cameras generates terabytes of data per hour. Uploading all this raw footage to the cloud over an industrial internet connection is prohibitively expensive and congests the network.
• Reliability: If the internet connection drops, a cloud-dependent manufacturing robot stops working entirely, halting production.

Edge computing solves the Cloud bottleneck by moving data processing, storage, and analytics as close as possible to the 'Edge' of the network—where the data is actually being generated by the sensors.

⇒2.1 How it Works

Instead of sending raw vibration data to the cloud, a local Edge device (like an industrial Raspberry Pi or a localized micro-server) runs an AI model right on the factory floor. The Edge device continuously analyzes the vibration. It only contacts the cloud to send a tiny, aggregated alert message: 'Bearing #4 shows a 90% probability of failure within 48 hours'.

This approach virtually eliminates latency, drastically reduces bandwidth costs (sending a 100-byte alert instead of gigabytes of raw data), and allows the system to continue operating even if the internet goes down.

These terms are often used interchangeably, but Cisco originally defined a subtle architectural difference.

⇒3.1 Edge Computing

Processing occurs strictly on the device itself or an immediate local gateway (e.g., the processor built directly into a smart security camera analyzing the video feed).

⇒3.2 Fog Computing

A broader, hierarchical concept. Fog computing pushes cloud-like services (compute, storage, networking) down to the Local Area Network (LAN) level. Processing might happen at a cell tower base station, an industrial network switch, or a neighborhood distribution node. It creates a 'fog' of decentralized compute power between the sharp 'edge' of the devices and the distant 'cloud'.

While Edge computing handles real-time reflexes, the Cloud remains essential for heavy lifting, long-term memory, and global management.

⇒4.1 Device Management and Digital Twins

IoT Cloud Platforms (like AWS IoT Core or Azure IoT Hub) provide massive infrastructure to securely register, authenticate, and manage millions of deployed devices.

A key concept is the Device Shadow (or Digital Twin). The cloud maintains a virtual JSON document representing the last known state of the physical device. If a mobile app wants to turn on a smart bulb, it doesn't talk to the bulb; it updates the 'desired state' in the Cloud's Device Shadow. When the bulb wakes up, it checks its Shadow, sees the new desired state, turns itself on, and updates the 'reported state'. This elegantly handles devices that are frequently offline or asleep.

Once the data reaches the cloud, it must be stored and analyzed. IoT data is almost universally Time-Series Data (a value, a sensor ID, and a precise timestamp).

⇒5.1 Storage Solutions

Traditional relational databases (SQL) buckle under the constant, high-velocity insert rate of millions of sensors. IoT architectures typically use:

• Time-Series Databases (TSDB): Like InfluxDB or TimescaleDB. These are heavily optimized for ingesting time-stamped data streams and quickly querying ranges (e.g., 'give me the average temperature over the last 30 days grouped by hour').
• Data Lakes: Cheap, massive storage (like AWS S3) for dumping raw, unstructured telemetry data for future machine learning training.

Data doesn't just sit in a database; it flows through a pipeline.

⇒6.1 Stream Processing

Before data even hits a database, it often passes through a stream processing engine (like Apache Kafka or AWS Kinesis). These engines act as massive buffers, absorbing sudden spikes in traffic (e.g., millions of devices coming back online after a power outage) and allowing analytics algorithms to evaluate the data 'in-flight' to trigger instant alarms before the data is permanently stored.

This unit shifts focus from how data moves to where it is processed. The shift from centralized to decentralized architecture is the core theme.

• Edge vs Cloud: You must be able to clearly articulate the three reasons we need Edge computing: Latency reduction, Bandwidth conservation, and Offline Reliability.
• Device Shadows: Understand how Digital Twins decouple the user interface from the physical device, solving the problem of sleepy/offline nodes.
• Time-Series Data: Explain why standard RDBMS architectures fail for IoT workloads, and why TSDBs or NoSQL solutions are preferred for massive telemetry ingestion.