Geode Pattern: Partition Data Across Independent Nodes

Overview

The Geode Pattern splits data into non-overlapping shards and hands each one to a node that owns it outright — storage included. A request for a given item goes straight to the node holding it, which answers from its own data. Nothing is shared: no central database, no common cache, no coordinator in the hot path.

Microsoft’s Azure Architecture Center documents this pattern for exactly the problem it solves: when every node reads and writes through the same backend, that backend becomes the ceiling. Move the data into the nodes and each one only carries its fraction of the load, so capacity grows roughly linearly as you add machines. The name is a good metaphor: crack a geode open and you find a self-contained cavity lined with crystals — same idea here, every node carries its own data inside.

What separates this from plain database sharding is the degree of independence. Sharding splits one logical database across several instances that still answer to the same engine. A geode goes further: each node is a fully independent unit that can run its own store and fail on its own terms.

When to Use

Use the Geode Pattern when:

• A shared database or cache is the scalability bottleneck
• Data has a natural partition key (user ID, tenant ID, region)
• You need fault isolation: one partition going down shouldn’t affect the rest
• You want horizontal scaling without distributed transactions

Typical fits: multi-tenant SaaS, game servers, IoT ingestion, regional content delivery.

Avoid it when your workload is dominated by cross-partition queries or distributed transactions — the coordination cost eats the partitioning benefit. For plain dataset splitting where nodes can still share an engine, the Sharding Pattern is simpler.

Solution

Each key hashes into a numeric range; each node owns a range. The cluster routes on the hash, and a node failure only hurts its own shard.

Python

from dataclasses import dataclass, field
from typing import Dict, Any, List, Optional
import hashlib

@dataclass
class GeodeNode:
    node_id: str
    shard_range: tuple
    data: Dict[str, Any] = field(default_factory=dict)
    healthy: bool = True

    def owns_key(self, key: str) -> bool:
        h = int(hashlib.md5(key.encode()).hexdigest(), 16) % 10000
        return self.shard_range[0] <= h <= self.shard_range[1]

    def get(self, key: str) -> Optional[Any]:
        if not self.healthy:
            raise RuntimeError(f"Node {self.node_id} is down")
        return self.data.get(key)

    def put(self, key: str, value: Any) -> None:
        if not self.healthy:
            raise RuntimeError(f"Node {self.node_id} is down")
        self.data[key] = value

class GeodeCluster:
    def __init__(self, num_nodes: int = 4):
        shard_size = 10000 // num_nodes
        self.nodes: List[GeodeNode] = [
            GeodeNode(
                node_id=f"node-{i}",
                shard_range=(
                    i * shard_size,
                    i * shard_size + shard_size - 1
                    if i < num_nodes - 1 else 9999,
                ),
            )
            for i in range(num_nodes)
        ]

    def find_node(self, key: str) -> Optional[GeodeNode]:
        return next((n for n in self.nodes if n.owns_key(key)), None)

    def get(self, key: str) -> Optional[Any]:
        node = self.find_node(key)
        if node is None:
            raise KeyError(f"No node owns key: {key}")
        return node.get(key)

    def put(self, key: str, value: Any) -> None:
        node = self.find_node(key)
        if node is None:
            raise KeyError(f"No node owns key: {key}")
        node.put(key, value)

    def fail_node(self, node_id: str) -> None:
        for n in self.nodes:
            if n.node_id == node_id:
                n.healthy = False

    def stats(self) -> List[dict]:
        return [{"node_id": n.node_id, "keys": len(n.data),
                 "healthy": n.healthy, "range": n.shard_range}
                for n in self.nodes]

# Usage
cluster = GeodeCluster(num_nodes=4)
users = ["alice", "bob", "carol", "dave", "eve", "frank", "grace", "heidi"]
for u in users:
    cluster.put(u, {"email": f"{u}@example.com", "score": len(u) * 10})

for s in cluster.stats():
    print(f"  {s}")

print("\n=== Simulate node-2 failure ===")
cluster.fail_node("node-2")
for u in users:
    node = cluster.find_node(u)
    try:
        cluster.get(u)
        print(f"  {u} -> {node.node_id}: OK")
    except RuntimeError as e:
        print(f"  {u} -> {node.node_id}: FAILED ({e})")

JavaScript

const crypto = require('crypto');

class GeodeNode {
  constructor(nodeId, shardStart, shardEnd) {
    this.nodeId = nodeId;
    this.shardRange = [shardStart, shardEnd];
    this.data = new Map();
    this.healthy = true;
  }

  ownsKey(key) {
    const h = parseInt(
      crypto.createHash('md5').update(key).digest('hex'), 16
    ) % 10000;
    return h >= this.shardRange[0] && h <= this.shardRange[1];
  }

  get(key) {
    if (!this.healthy) throw new Error(`Node ${this.nodeId} is down`);
    return this.data.get(key);
  }

  put(key, value) {
    if (!this.healthy) throw new Error(`Node ${this.nodeId} is down`);
    this.data.set(key, value);
  }
}

class GeodeCluster {
  constructor(numNodes = 4) {
    this.nodes = [];
    const shardSize = Math.floor(10000 / numNodes);
    for (let i = 0; i < numNodes; i++) {
      const start = i * shardSize;
      const end = i < numNodes - 1 ? start + shardSize - 1 : 9999;
      this.nodes.push(new GeodeNode(`node-${i}`, start, end));
    }
  }

  findNode(key) {
    return this.nodes.find(n => n.ownsKey(key)) || null;
  }

  get(key) {
    const node = this.findNode(key);
    if (!node) throw new Error(`No node owns key: ${key}`);
    return node.get(key);
  }

  put(key, value) {
    const node = this.findNode(key);
    if (!node) throw new Error(`No node owns key: ${key}`);
    node.put(key, value);
  }

  failNode(nodeId) {
    const node = this.nodes.find(n => n.nodeId === nodeId);
    if (node) node.healthy = false;
  }

  stats() {
    return this.nodes.map(n => ({
      nodeId: n.nodeId, keys: n.data.size,
      healthy: n.healthy, range: n.shardRange,
    }));
  }
}

// Usage
const cluster = new GeodeCluster(4);
const users = [
  "alice", "bob", "carol", "dave", "eve", "frank", "grace", "heidi",
];
users.forEach(u =>
  cluster.put(u, { email: `${u}@example.com`, score: u.length * 10 }));

cluster.stats().forEach(s => console.log("  ", s));

console.log("\n=== Simulate node-2 failure ===");
cluster.failNode("node-2");
users.forEach(u => {
  const node = cluster.findNode(u);
  try {
    cluster.get(u);
    console.log(`  ${u} -> ${node.nodeId}: OK`);
  } catch (e) {
    console.log(`  ${u} -> ${node.nodeId}: FAILED (${e.message})`);
  }
});

Java

import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.*;

public class GeodeCluster {

    static class GeodeNode {
        final String nodeId;
        final int shardStart, shardEnd;
        final Map<String, Object> data = new HashMap<>();
        boolean healthy = true;

        GeodeNode(String nodeId, int shardStart, int shardEnd) {
            this.nodeId = nodeId;
            this.shardStart = shardStart;
            this.shardEnd = shardEnd;
        }

        boolean ownsKey(String key) {
            int h = hashKey(key);
            return h >= shardStart && h <= shardEnd;
        }

        Object get(String key) {
            if (!healthy)
                throw new RuntimeException("Node " + nodeId + " is down");
            return data.get(key);
        }

        void put(String key, Object value) {
            if (!healthy)
                throw new RuntimeException("Node " + nodeId + " is down");
            data.put(key, value);
        }
    }

    final List<GeodeNode> nodes = new ArrayList<>();

    public GeodeCluster(int numNodes) {
        int shardSize = 10000 / numNodes;
        for (int i = 0; i < numNodes; i++) {
            int start = i * shardSize;
            int end = (i < numNodes - 1) ? start + shardSize - 1 : 9999;
            nodes.add(new GeodeNode("node-" + i, start, end));
        }
    }

    static int hashKey(String key) {
        try {
            byte[] d = MessageDigest.getInstance("MD5")
                .digest(key.getBytes(StandardCharsets.UTF_8));
            int h = 0;
            for (int i = 0; i < 4; i++) h = (h << 8) | (d[i] & 0xFF);
            return Math.abs(h) % 10000;
        } catch (Exception e) {
            return Math.abs(key.hashCode()) % 10000;
        }
    }

    GeodeNode findNode(String key) {
        return nodes.stream().filter(n -> n.ownsKey(key))
            .findFirst().orElse(null);
    }

    Object get(String key) {
        GeodeNode node = findNode(key);
        if (node == null)
            throw new RuntimeException("No node owns key: " + key);
        return node.get(key);
    }

    void put(String key, Object value) {
        GeodeNode node = findNode(key);
        if (node == null)
            throw new RuntimeException("No node owns key: " + key);
        node.put(key, value);
    }

    void failNode(String nodeId) {
        nodes.stream()
            .filter(n -> n.nodeId.equals(nodeId))
            .forEach(n -> n.healthy = false);
    }

    public static void main(String[] args) {
        var cluster = new GeodeCluster(4);
        String[] users = {"alice", "bob", "carol", "dave",
                          "eve", "frank", "grace", "heidi"};
        for (String u : users)
            cluster.put(u, Map.of(
                "email", u + "@example.com", "score", u.length() * 10));

        System.out.println("=== Simulate node-2 failure ===");
        cluster.failNode("node-2");
        for (String u : users) {
            var node = cluster.findNode(u);
            try {
                cluster.get(u);
                System.out.printf("  %s -> %s: OK%n", u, node.nodeId);
            } catch (Exception e) {
                System.out.printf("  %s -> %s: FAILED (%s)%n",
                    u, node.nodeId, e.getMessage());
            }
        }
    }
}

Explanation

The mechanics are three steps:

1. Partition by hash. Every key hashes to a number inside a fixed range (here 0-9999). Each node owns a contiguous, non-overlapping slice, so exactly one node is responsible for any key.
2. Route on the hash. Whatever receives the request — an edge router, a client library, the cluster itself — recomputes the hash and sends the request to the owning node. The node answers from local state, with no shared backend in the way.
3. Fail in isolation. A dead node only takes its own shard down. The rest keep serving, and the blast radius is one node’s worth of keys.

The demo above simulates step 3: kill node-2 and only its users fail, while the rest answer normally.

Rebalancing and failover

Fixed ranges like the example are easy to reason about but painful to change: adding a node means slicing ranges and migrating keys. Two standard answers:

• Consistent hashing puts nodes and keys on a ring. Adding or removing a node only remaps the slice between it and its neighbors, so most keys stay put — this is why Dynamo-style systems use it.
• Virtual nodes give each physical node several small ranges. Load spreads more evenly, and rebalancing moves a few virtual slices instead of one giant shard.

For failover, replicate each shard to a second node (read-only is usually enough) and promote on failure. Without a replica, a dead node’s keys are simply unavailable until it comes back — acceptable for caches, unacceptable for the source of truth.

Variants

Variant	Description	Use case
Consistent hashing	Ring instead of fixed ranges for cheap rebalancing	Dynamic clusters where nodes join and leave
Replicated geodes	Each shard copied to a second node for failover	High availability without shared state
Geographic geodes	Partition by region, each in its own datacenter	Latency-sensitive global apps
Virtual nodes	One node owns several non-contiguous ranges	Better balance with few physical nodes

What Works

• Partition by a natural key (user ID, tenant ID): related data lands on the same node and queries stay local.
• Prefer consistent hashing once the cluster is live — fixed ranges make every topology change a migration project.
• Keep shards independent: the moment two shards coordinate on a write, you’ve rebuilt the shared-state bottleneck you were escaping.
• Watch shard balance: a hot key or a skewed distribution silently concentrates load on one node. Track keys and traffic per shard and rebalance when one drifts.
• Route at the edge: a gateway or client library owns the key-to-node mapping so callers stay oblivious to the topology.
• Replicate critical shards, keeping replicas read-only to avoid a coordination protocol.

Common Mistakes

• Uneven partition keys — a celebrity user or a huge tenant turns one shard into a hotspot while the rest idle.
• Distributed transactions across shards — the coordination overhead negates the whole point of partitioning.
• No failover plan — a node dies and its slice of data is gone until someone notices.
• Cross-shard joins in the hot path — scatter-gather works for occasional reads, not for every request.
• Unmonitored shard growth — one shard quietly fills its disk while aggregate usage looks fine.
• Random partitioning — without a natural key you lose locality and end up scatter-gathering for routine lookups.

Further Reading

• Geode pattern — Azure Architecture Center
• Sharding pattern — Azure Architecture Center
• Dynamo: Amazon’s Highly Available Key-value Store (consistent hashing)