NoSQL Database Selection — MongoDB, DynamoDB, Cassandra

Introduction

NoSQL databases trade the strict consistency and relational model of SQL for flexibility, horizontal growth, and specialized access patterns. Choosing the right one means matching your data shape, query patterns, and consistency requirements to the right store.

The Four NoSQL Families

Family	Structure	Best For	Examples
Document	JSON-like documents with nested structures	Content management, user profiles, catalogs	MongoDB, Firestore, Couchbase
Key-Value	Simple key → value lookups	Sessions, caching, feature flags	Redis, DynamoDB, Riak
Wide-Column	Column families with rows as sparse maps	Time-series, high-write telemetry, messaging	Cassandra, HBase, ScyllaDB
Graph	Nodes and relationships with properties	Social networks, recommendation engines, fraud detection	Neo4j, Amazon Neptune

One thing all of them share: tunable consistency. Cassandra, MongoDB, and DynamoDB let you dial the trade-off per operation — write to one replica for speed (W=1, R=1), require a quorum for correctness (W+R>N), or pay for strict reads (DynamoDB ConsistentRead, MongoDB majority). Pick the store first, but tune the consistency knobs deliberately.

A Selection Process That Works

Most teams pick a NoSQL database backwards — they start with a product name, then bend their data to fit. The order that avoids rewrites:

1. List your access patterns, not your entities. “Get user by ID, get posts by author sorted by date, count unread notifications” — that list is what the database actually serves. Entities change; access patterns don’t.
2. Eliminate by hard constraint. Does all data fit in RAM? → Redis viable, but not as primary storage. Are queries mostly relationship traversals? → graph store. Is the write rate thousands per second sustained? → wide-column. Most choices die here, before any benchmark.
3. Prototype the top two candidates with your real key distribution — Zipf-shaped, not uniform random. A store that looks flat under uniform load often falls apart under real skew.
4. Decide on operational burden. DynamoDB has no cluster to babysit; Cassandra needs a team that understands repairs, compaction, and gossip. A managed service premium is usually cheaper than the on-call rotation.

The decision matrix later in this guide compresses step 2. For step 3, see the benchmarking section — synthetic tests lie more often than they tell the truth.

Document Stores: MongoDB

When to Choose

• Rich, nested data structures with arrays and subdocuments
• Flexible schema that evolves over time
• Need for secondary indexes and aggregation pipelines
• Queries that look like JavaScript object matching

Example

// A product document with nested reviews and variants
db.products.insertOne({
  sku: "SHOE-001",
  name: "Trail Runner",
  price: 89.99,
  attributes: { color: "red", size: 42 },
  reviews: [
    { user_id: 42, rating: 5, comment: "Great grip!" }
  ],
  tags: ["running", "trail", "waterproof"]
})

// Flexible query with nested matching
db.products.find({ "reviews.rating": { $gte: 4 }, tags: "trail" })

Trade-offs

Pro	Con
Flexible schema	Schema validation must be configured explicitly
Rich query language	Joins are expensive and limited
Secondary indexes	Indexes consume RAM and slow writes
Horizontal scaling (sharding)	Sharding adds operational complexity

Cost and Operations

• Working set must fit in RAM — if hot data and its indexes exceed memory, MongoDB hits disk and latency climbs. Size for the working set, not the dataset.
• Indexes cost writes — every index adds work to each insert; collections with heavy write load need index discipline.
• The shard key is permanent — pick wrong (monotonically increasing, low cardinality) and you’ll rebalance a live cluster later.
• MongoDB Atlas vs self-hosted: Atlas removes ops burden (backups, upgrades, monitoring) but costs a premium. Self-hosting sharded clusters requires real operational expertise.

Key-Value Stores: DynamoDB and Redis

DynamoDB (AWS)

Best for: predictable latency at any scale, simple read/write patterns, serverless architectures.

import boto3
from boto3.dynamodb.conditions import Key

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Users')

# Get by partition key (single-digit ms latency)
table.get_item(Key={'user_id': 'user-123'})

# Query by partition + sort key
table.query(
    KeyConditionExpression=Key('user_id').eq('user-123') &
                           Key('timestamp').gt('2024-01-01')
)

Critical design constraint: Access patterns must be known upfront. DynamoDB is optimized for known query paths, not ad-hoc exploration.

Redis

Best for: caching, real-time leaderboards, rate limiting, session stores.

# Cache a computed value for 5 minutes
SET user:123:profile '{"name":"Alice"}' EX 300

# Atomic increment for rate limiting
INCR rate_limit:ip:192.168.1.1
EXPIRE rate_limit:ip:192.168.1.1 60

Critical constraint: All data must fit in RAM. Redis isn’t a primary data store for large datasets.

Cost and Operations

• DynamoDB pricing is per-request — on-demand mode charges per RCU/WCU consumed; provisioned mode is cheaper at steady load but needs capacity planning. Item size matters: a 4KB write costs 4× a 1KB write.
• No cluster to manage — backups (point-in-time recovery), encryption, and scaling are built in. The operational cost is near zero; the design cost is high.
• Redis is single-threaded — one instance handles ~100K+ ops/sec, but your dataset plus replication headroom must fit in the node’s RAM. Redis Cluster adds sharding at the price of restricted multi-key operations.

Wide-Column Stores: Cassandra

When to Choose

• Write-heavy workloads (time-series, IoT, messaging — for dedicated engines see the time-series databases guide)
• Need linear growth across commodity hardware
• Tolerance for eventual consistency and CQL (Cassandra Query Language)

Data Model

-- Time-series sensor data
CREATE TABLE sensor_readings (
    sensor_id UUID,
    day TEXT,        -- partition key component
    timestamp TIMESTAMP,
    temperature DOUBLE,
    humidity DOUBLE,
    PRIMARY KEY ((sensor_id, day), timestamp)
) WITH CLUSTERING ORDER BY (timestamp DESC);

-- Query: last 100 readings for a sensor today
SELECT * FROM sensor_readings
WHERE sensor_id = ? AND day = '2024-06-12'
LIMIT 100;

Cassandra is query-first: tables are designed around specific read queries, not normalized entities.

Trade-offs

Pro	Con
Massive write throughput	No JOINs, no subqueries, no aggregations across partitions
Linear growth	Operational complexity (gossip, repairs, compaction)
Multi-datacenter replication	Eventual consistency by default
Tunable consistency	CQL is limited compared to SQL

Cost and Operations

• Repairs are mandatory — skipping them lets data diverge silently between replicas. Expect a standing nodetool repair schedule or a managed service that handles it.
• Compaction affects everything — the strategy (STCS, LCS, TWCS) changes read/write amplification; the wrong one burns I/O.
• JVM operational expertise — GC pauses, heap tuning, and node sizing are real work. If the team is small, look at ScyllaDB (Cassandra-compatible, self-managed internals) or Amazon Keyspaces (managed, CQL-compatible).
• Minimum viable cluster is 3 nodes — per datacenter. Multi-DC active-active is Cassandra’s killer feature, but each datacenter is a full cluster.

Graph Stores: Neo4j

Best for: data where the relationships are the value — social graphs, fraud rings, dependency analysis.

// Friend-of-friend recommendations
MATCH (me:User {id: "user-123"})-[:FOLLOWS]->(:User)
      -[:FOLLOWS]->(rec:User)
WHERE NOT (me)-[:FOLLOWS]->(rec) AND me <> rec
RETURN rec.id, rec.name, count(*) AS mutual_friends
ORDER BY mutual_friends DESC LIMIT 10;

Critical constraint: graph stores shine at traversal depth; they underperform as general-purpose OLTP. If your queries are mostly single-entity reads, a document or key-value store does the job cheaper.

Cost and Operations

• The hot graph must fit in RAM — Neo4j’s page cache determines query speed; traversals that spill to disk degrade quickly.
• Write scaling is limited — Neo4j clusters replicate for read scale and availability, but writes go through the leader. Write-heavy graphs are a poor fit.
• Aura managed offering — Neo4j Aura removes cluster ops (backups, upgrades, failover) at a subscription premium; self-hosting means owning JVM tuning and backup scripts.

Decision Matrix

Requirement	Best Choice	Why
Flexible, nested JSON documents	MongoDB	Native document model, rich query language
Predictable low-latency key lookups at scale	DynamoDB	Single-digit ms, auto-scaling, serverless
High-throughput time-series writes	Cassandra	Log-structured storage, excellent write performance
Caching and ephemeral data	Redis	In-memory speed, rich data structures
Complex relationship traversal	Neo4j	Optimized graph traversals
Multi-item ACID transactions	PostgreSQL	NoSQL stores typically lack cross-document transactions
Real-time leaderboards and sorted data	Redis	Sorted sets (ZADD/ZRANGEBYSCORE) are native structures
Full-text search over documents	Elasticsearch	Inverted index built for search — not a primary store
Time-series telemetry at scale	Cassandra or a dedicated TSDB	Wide-column handles write volume; TSDBs add retention policies
Event sourcing / immutable logs	DynamoDB or Cassandra	Append-only partitions with time-ordered keys

Scaling and Clustering Basics

Each family scales differently, and the cluster model determines both the ceiling and the ops burden:

Store	Replication Model	Scale Ceiling	Cluster Ops
MongoDB	Leader + replica sets, sharded	~50 shards typical	Medium — config servers, mongos routing
DynamoDB	Fully managed multi-AZ	Practically unlimited	None — no cluster visible
Redis	Leader + replicas, Redis Cluster	1000 shards	Low-Medium — slot resharding
Cassandra	Leaderless, multi-DC	Thousands of nodes	High — gossip, repairs, compaction
Neo4j	Leader + followers	Single-graph size	Medium — backup/restore discipline

Two patterns worth knowing upfront:

• Leader-based (MongoDB, Redis, Neo4j): one node accepts writes, others replicate. Simple reasoning about consistency; the leader is a write bottleneck and a failover event.
• Leaderless (Cassandra): any node accepts writes and coordinates with the rest — no failover ceremony, but you manage consistency levels and conflict resolution per query.

DynamoDB hides all of this behind a managed abstraction — the “cluster” exists but AWS runs it. That’s the real trade: operational simplicity in exchange for per-request pricing and design constraints.

Cross-region is the other axis: Cassandra was built for active-active multi-datacenter writes (write in EU and US simultaneously, converge later). DynamoDB Global Tables gives you the same pattern fully managed. MongoDB and Neo4j can replicate across regions but keep a single writer, so far users either tolerate write latency or get routed to a closer primary. If your app writes from two continents, this constraint eliminates half the field before benchmarks even start.

Migration Tips from SQL

SQL Habit	NoSQL Adaptation
Normalized tables	Embed related data when accessed together; reference when accessed separately
JOINs everywhere	Design tables/collections around query patterns, not entities
Auto-increment IDs	Use UUIDs or composite keys (user_id + timestamp)
Ad-hoc analytics	Use change data capture to stream to a data warehouse
Single source of truth	Accept that different stores may have different views of truth (CQRS)

What Works

• Model for your reads, not your writes — NoSQL performance is access-pattern dependent
• Avoid hot partitions — distribute writes evenly across partition keys (use random suffixes or time bucketing)
• Set TTLs where appropriate — expire old data automatically instead of running cleanup jobs
• Test with production-like data volumes — behavior at 1K rows is not predictive of behavior at 1B rows
• Have a migration path — data gravity is real; choose carefully because migrating later is expensive

Common Mistakes

• Using MongoDB as a cache (Redis is cheaper and faster)
• Using DynamoDB for ad-hoc analytics (Athena/BigQuery are better suited)
• Using Cassandra for OLTP with complex queries (Cassandra excels at simple, partition-scoped queries)
• Treating NoSQL as “grows better SQL” — the data model works differently
• Ignoring operational complexity — Cassandra and sharded MongoDB require dedicated operational expertise

Troubleshooting

• A few partitions do all the work (hot partitions): skewed partition keys concentrate load on a few nodes — add a random suffix or time bucket to the key, and check per-partition metrics (DynamoDB Contributor Insights, Cassandra nodetool cfstats).
• Reads get slower as deleted data piles up (Cassandra tombstones): heavy deletes leave tombstones that scans must skip — align gc_grace_seconds with your repair cadence and prefer TTL expirations over explicit DELETEs.
• DynamoDB queries throttle despite low overall volume: on-demand vs provisioned mismatch, or a hot partition key exceeding the 1000 WCU/3000 RCU per-partition limit — inspect CloudWatch ThrottledRequests by key.
• MongoDB aggregation crawls on a big collection: the pipeline scans without an index — reorder so $match/$sort stages come first and verify with explain("executionStats").
• Cross-store data drifts apart: in polyglot setups, a failed consumer leaves stores inconsistent — reconcile via the event log (replay from Kafka) or periodic diff jobs rather than fixing rows by hand.

Benchmarking Your Choice

Synthetic benchmarks mislead more than they help — YCSB with uniform keys flatters every store. To make the comparison honest:

• Use your real key distribution. Production traffic is Zipf-shaped: a few hot keys, a long tail. Test with skewed data or your results mean nothing.
• Measure p99, not averages. A store averaging 5ms reads can still deliver 500ms tail latencies that break your SLA — and tail behavior is where consistency knobs show their cost.
• Kill a node mid-test. Every NoSQL database looks fine until a replica dies. Measure throughput and latency during failure, not just at steady state.
• Test your top 3 access patterns — the queries your app actually runs 95% of the time — not a generic workload mix.
• Include the ops surface in the scorecard. A 20% throughput advantage means little if the cluster needs a dedicated SRE.

Finally, run the same test twice — once on the managed version, once self-hosted if you’re weighing both. Managed DynamoDB vs self-hosted Cassandra, or Atlas vs your own MongoDB replica set, often flips the comparison once pricing and paging hours enter the equation. A store that loses the benchmark but wins on total cost of ownership is usually the right call.

Advanced Topics

Scenario: Polyglot Persistence for Social Platform

System: 50M users, 200M posts, 2B interactions/day
Stores: 6 databases, each optimized for its workload

Store selection:
  | Data | Store | Why | Access Pattern |
  |------|-------|-----|----------------|
  | Profiles | MongoDB | Flexible schema | Point reads by user_id |
  | Posts | Cassandra | High write volume | Partition by user_id |
  | Graph | Neo4j | Relationship traversal | Friend-of-friend |
  | Feed | Redis | Low-latency sorted sets | ZREVRANGE |
  | Search | Elasticsearch | Full-text, faceted | Multi-field search |
  | Analytics | ClickHouse | Columnar OLAP | Aggregations |

Data flow:
  User creates post
    -> Write to Cassandra (durable)
    -> Publish event to Kafka
    -> Consumers:
       a. Feed: ZADD to Redis (fan-out to followers)
       b. Search: Index in Elasticsearch
       c. Analytics: Insert into ClickHouse
       d. Graph: Update Neo4j relationships

  Cassandra schema:
    CREATE TABLE posts_by_user (
        user_id UUID, post_id TIMEUUID, content TEXT,
        media_urls LIST<TEXT>, created_at TIMESTAMP,
        PRIMARY KEY ((user_id), post_id)
    ) WITH CLUSTERING ORDER BY (post_id DESC);

  Redis timeline (fan-out on write):
    ZADD feed:user:{follower_id} {timestamp} {post_id}
    EXPIRE feed:user:{follower_id} 86400
    ZREVRANGE feed:user:{user_id} 0 49 WITHSCORES

  Neo4j social graph:
    MATCH (me:User {id: "user-123"})-[:FOLLOWS]->(:User)
          -[:FOLLOWS]->(rec:User)
    WHERE NOT (me)-[:FOLLOWS]->(rec) AND me <> rec
    RETURN rec.id, rec.name, count(*) AS mutual_friends
    ORDER BY mutual_friends DESC LIMIT 10;

  ClickHouse analytics:
    CREATE TABLE interactions (
        event_date Date, user_id UInt64, post_id UInt64,
        event_type Enum8("like"=1,"comment"=2,"share"=3,"view"=4),
        timestamp DateTime
    ) ENGINE = MergeTree()
    PARTITION BY toYYYYMM(event_date)
    ORDER BY (event_type, timestamp);

Operational complexity:
  | Challenge | Mitigation |
  |-----------|------------|
  | 6 databases | Dedicated team per store |
  | Cross-store consistency | Outbox + Kafka events |
  | Schema evolution | Versioned events |
  | Data duplication | Accept; each store optimized |
  | Monitoring | Unified dashboard per store |

Lessons:
  - Polyglot persistence is capable but expensive
  - Start with 1-2 stores, add more when patterns diverge
  - Event-driven sync (Kafka) is the glue
  - Each store owns a specific access pattern
  - Data duplication is acceptable; manage via events

The event flow from that scenario, visualized:

See Also

• NoSQL Data Modeling Patterns — embed vs reference, access-pattern-driven design
• CAP Theorem Guide — the consistency/availability trade-off behind every store above
• MongoDB documentation — aggregation pipeline, sharding, indexes
• Amazon DynamoDB documentation — partition design, GSI, capacity modes
• Apache Cassandra documentation — data modeling, tunable consistency, tombstones
• Companion repo — runnable schemas and examples for all four stores