Scalability Basics
18 min
Horizontal scaling means spreading data (or load) across multiple machines instead of making one machine bigger. Sharding is how you decide which machine owns which piece of data. The simplest strategy is range-based sharding: divide the key space into contiguous ranges, one per shard — like assigning user IDs 0-999 to shard 0, 1000-1999 to shard 1, and so on.
def shard_for_key(key, boundaries):
for i, boundary in enumerate(boundaries):
if key <= boundary:
return i
return len(boundaries)
Range sharding's big advantage is that a RANGE QUERY like give me everything between key 100 and 250 only needs to touch 1-2 shards -- unlike hash-based sharding, where a range query would need to scatter across every single shard, since a hash deliberately destroys any relationship between nearby keys.
Write `shard_for_key(key, boundaries)`: `boundaries` is a sorted list of upper-bound values, one per shard except the last (unbounded) shard. Return the index of the first shard whose boundary is `>= key`, or the last shard's index if `key` exceeds every boundary.
What's the main weakness of RANGE-based sharding (like `shard_for_key`) compared to hash-based sharding?
You can implement range-based sharding and understand its core tradeoff against hash-based sharding: efficient range queries vs. even load distribution.