Load Balancing
18 min
A load balancer sits in front of a pool of servers and decides which one handles each incoming request. The simplest strategy is round robin: cycle through the pool in order, spreading requests evenly.
def round_robin_assignments(servers, num_requests):
assignments = []
for i in range(num_requests):
assignments.append(servers[i % len(servers)])
return assignments
The modulo (%) is doing all the work here: it wraps the ever-increasing request index back into range every time it passes the end of the server list -- the same wraparound trick used for circular buffers and hash table probing.
Plain round robin assumes every server can handle the same load. In practice, servers often differ in capacity — one instance might have double the CPU of another. Weighted round robin fixes this by giving each server a share of requests proportional to its weight: a simple way to implement it is to expand the server list by each server's weight, then round-robin over the EXPANDED list.
Write `round_robin_assignments(servers, num_requests)`: return a list of length `num_requests` cycling through `servers` in order (request 0 goes to servers[0], request 1 to servers[1], wrapping back to servers[0] after the last server).
Write `weighted_assignments(servers_weights, num_requests)`: `servers_weights` is a list of `(server, weight)` pairs. A server with weight 2 should receive twice as many requests as a server with weight 1, still assigned round-robin overall.
Why would a load balancer use WEIGHTED round-robin instead of plain round-robin?
You can implement round-robin and weighted round-robin request distribution, and explain why capacity differences between servers motivate weighting.