NoSQL stores and data modelling notes — Unit 4

For decades, Relational Database Management Systems (RDBMS) utilizing SQL and rigid tabular schemas were the absolute standard for data storage. However, the Big Data era exposed their limitations.

⇒1.1 The RDBMS Bottleneck

RDBMS are designed to scale vertically (buying a bigger server), which has physical limits and high costs. They struggle with unstructured data and require rigid schema definitions before data can be inserted. NoSQL ('Not Only SQL') databases were created to scale horizontally across commodity hardware and handle schema-less data.

Unlike RDBMS, which is a single paradigm, NoSQL is an umbrella term encompassing four primary architectural types, each optimized for different use cases.

• Key-Value Stores
• Document Stores
• Column-Family Stores
• Graph Databases

The simplest type of NoSQL database. Every item in the database is stored as an attribute name (or 'key'), together with its value.

⇒3.1 Characteristics

The database treats the 'value' as a complete black box. It cannot query inside the value (e.g., 'Find all users where age > 30' is impossible if the age is hidden inside the value blob). You can only retrieve data if you know the exact Key.

• Pros: Blazing fast reads/writes; infinitely horizontally scalable.
• Use Cases: User session data, shopping carts, caching.
• Examples: Redis, Amazon DynamoDB.

Similar to Key-Value, but the 'Value' is a structured document (usually JSON or BSON).

⇒4.1 Characteristics

Because the database understands the structure of the JSON document, you can query inside it. They are schema-less; Document A can have a 'Name' and 'Age' field, while Document B in the same collection has 'Name', 'Location', and an array of 'Hobbies'.

• Pros: Flexible schema, maps perfectly to object-oriented programming.
• Use Cases: Content management systems, product catalogs, user profiles.
• Examples: MongoDB, CouchDB.

Designed for storing massive amounts of data distributed over many machines.

⇒5.1 Characteristics

Instead of storing data as rows, data is stored as sections of columns. This allows for massive compression and incredibly fast queries when analyzing specific columns across billions of records (e.g., calculating the average temperature from a billion sensor readings).

• Pros: Exceptional performance for write-heavy workloads and wide-column queries.
• Use Cases: Time-series data, IoT sensor logs, massive event logging.
• Examples: Apache Cassandra, HBase.

Designed to store and navigate relationships. Data is stored in Nodes (entities) and Edges (relationships between entities).

⇒6.1 Characteristics

In an RDBMS, querying deep relationships requires multiple expensive `JOIN` operations. In a Graph DB, relationships are first-class citizens stored natively, making traversals instantaneous.

• Pros: Ideal for highly connected data.
• Use Cases: Social networks (friends of friends), fraud detection rings, recommendation engines.
• Examples: Neo4j, Amazon Neptune.

The CAP theorem (Consistency, Availability, Partition Tolerance) dictates that distributed NoSQL databases must make a trade-off.

⇒7.1 BASE vs ACID

Relational databases focus on ACID properties (Atomicity, Consistency, Isolation, Durability) guaranteeing absolute data integrity. They choose Consistency over Availability.

Many NoSQL databases embrace BASE semantics (Basically Available, Soft state, Eventual consistency). They prioritize high Availability and Partition Tolerance, accepting that data might be temporarily inconsistent across nodes.

Data modeling in NoSQL requires unlearning RDBMS habits.

⇒8.1 The Paradigm Shift

In SQL, you design the schema based on the data structure (Normalizing into multiple tables to reduce redundancy).

In NoSQL, you design the schema based on the application's access patterns (Query-driven design). You frequently Denormalize data (duplicate it) to ensure a single query can retrieve all necessary information without expensive JOINs.

• NoSQL Need: RDBMS struggle with horizontal scaling and unstructured data.
• Key-Value: Fastest, opaque values (Redis).
• Document: JSON schemas, flexible querying (MongoDB).
• Column-Family: Massive write throughput, time-series data (Cassandra).
• Graph: Native relationship traversal (Neo4j).
• Design: NoSQL relies on BASE semantics (Eventual Consistency) and Query-driven, denormalized data modeling.