Design a Chat System (WhatsApp / Messenger)

'Online' here means one narrow thing: your own phone can be reached by a server, right now. (Seeing which of your friends are online is a different feature, and we leave it for the end.) So the question is: what kind of channel lets a server put a message on your phone's screen the moment that message exists, without the phone asking for it?

There are three honest ways to get a message onto a phone: keep asking, ask and wait, or hold a line open.

Keep asking is what the day-one version already does — polling, just turned up. It has a wall built into it. To make delivery feel instant, the phone would have to ask many times a second, and almost every answer would still be 'nothing new.' So the request rate explodes while the useful work stays the same. And even then, there is always a gap between polls. You can pay more and more and never actually reach 'instant.'

Ask and wait is smarter. The phone sends a request, and the server simply does not answer yet. It holds the request open until a message actually exists, then replies with it. This is called long-polling, and it genuinely delivers promptly — real chat products shipped on it for years. But look at what it does: it holds a connection open for the wait, then tears it down after every single message and immediately opens a new one. At millions of messages a second, that is millions of teardowns and re-opens a second, for connections we wanted open the whole time.

Both dead ends point at the same fact. If a server must be able to reach a phone at any moment, something has to stay open between them the whole time. Once you accept that, the sensible move is to hold that connection once, openly, and stop paying to recreate it.

Persistent WebSocket. Every online phone opens one WebSocket and keeps it open. A WebSocket starts life as an ordinary web request carrying an Upgrade header. After the server accepts it, the connection does not close — it stays open and becomes two-way, which means the server can send data down it at any time, without being asked. That is exactly the property polling could never buy. A message for you arrives at the server, the server writes it down your socket, and it is on your screen. No asking, no gap.

Two housekeeping details make this real. First, a heartbeat: a tiny ping and pong travels over each connection every ten to thirty seconds, so the server notices when a phone has silently vanished — a dead battery, a lost signal — and cleans up. Second, the server itself changes character. It is no longer answering quick requests and forgetting you; its whole job is now holding your connection open. That makes it a different kind of server, so we rename it for what it now does: a chat server. The trade-off we accept is that a chat server is stateful — it holds particular people's live connections, so losing one matters. What that costs, and how we survive it, is the last use case.

Every online phone now holds one open connection, and those connections have to live somewhere. Of our billion users, roughly a fifth are online at any moment — two hundred million open connections. So before a single message moves: how many servers does it take just to hold them? That comes down to one number — how many connections we design each box to carry.

Start with the number everyone reaches for: about sixty-five thousand connections per server. It comes from the size of a port number, and it is a myth — that ceiling limits a client talking to one address, not a server accepting connections, because the server reuses its one listening port for every client. (The full story, from first principles, is under the hood in the last deep dive.) If you believed the myth, you would size a fleet of over three thousand boxes against a wall that is not there.

The real fork is between two believable numbers. WhatsApp proved that one tuned box can hold about two million connections. Design to that hero number and the fleet is only about a hundred boxes. Or design to something deliberately smaller — half a million per box — and run about four hundred.

The hero number has a catch: it is an idle ceiling. An open connection that is sitting quietly costs almost nothing — a handle and a few kilobytes of memory. Moving messages across it is what costs: every message wakes the server, gets copied through it, and gets written to exactly the right socket, and all of that burns CPU. Two million quiet connections fit on one box; two million busy ones do not. Pack a box to its idle ceiling and there is nothing left for the actual work.

Conservative — half a million per box. We design each chat server to hold five hundred thousand connections. That deliberately leaves most of the box's capacity free for processing messages, which is the load that really caps it. The arithmetic is then one line, and it is the line that defines chat at scale: two hundred million connections, at half a million per box, is about four hundred chat servers. Four hundred machines doing nothing but keeping sockets open, before one message is stored. We accept the bigger fleet on purpose — it is the price of headroom.

New connections need to be spread across that fleet, so a load balancer goes in front. When a phone connects, the balancer picks a chat server for it, and once the WebSocket is open, the connection stays on that server until it drops — 'sticky' just means the balancer places a reconnecting device predictably. What is the balancer itself? Unlike a chat server, it keeps no per-person state: it forwards a connection's bytes without reading them, which is such light work that a small set of balancer machines fronts the whole fleet, sharing one public address. Losing one balancer does drop the connections passing through it — but they reconnect through the survivors within seconds, and unlike a chat server, nothing about any person lived on the box.

A's chat server is holding A's message. B's socket is on one of four hundred servers, and nothing so far says which. How does A's server find the one server that holds B — for every message, three million times a second?

The crudest answer is to not find B at all: shout. A's server sends the message to all four hundred servers; the one holding B delivers it, and the other three hundred ninety-nine ignore it. No lookup, no bookkeeping to maintain — and every 1:1 message now costs four hundred times the work. At three million messages a second, that is over a billion server-to-server touches a second. It is ruled out on the spot — but worth saying out loud, because 'avoid the broadcast' is the requirement every real answer has to meet.

A much better idea: make B's location computable instead of stored. When a phone connects, choose its server by hashing its id — running the id through a fixed function that turns it into, in effect, a server number. Then any server can re-run the hash on B's id and know where B is, with no lookup at all. This idea is half right, and we keep the half that works. We do place connections this way, because it spreads people evenly across the fleet. But as the source of truth it has a flaw: the hash tells you where B should be, not where B is. B's assigned server was full, so B landed elsewhere. B's server died, so B reconnected somewhere else. B opened the app on a laptop — a second device the hash knows nothing about. The hash goes stale exactly when reality moves, and in chat, reality moves constantly.

Third idea: let the servers tell each other. Each chat server gossips who it is holding, and eventually every server knows where everyone is. Gossip is genuinely the right tool for a small, slowly-changing fact — like which of the four hundred servers are alive right now, and the fleet does use it for that. But per-user location is two hundred million facts that change on every connect and disconnect. Word of mouth lags, and here a lag has a concrete cost: someone who just connected is not yet known everywhere, so their first messages miss.

Follow all three failures and they point at the same requirement: a single place that always knows, updated at the exact moment the truth changes — when a socket opens or closes.

A connection directory. One shared lookup table: for every online device, which chat server holds its socket right now. The moment your phone's connection opens, your chat server writes your entry. The moment it closes, the entry is cleared. To route a message, A's server asks one question — where is B? — gets one answer — chat server 340 — and forwards the message there. Chat server 340 finds B's socket in its own in-memory table and writes the message down it. One lookup, one hop, done.

What is this table, concretely? It is a plain key-value lookup at a very high read rate, and the standard tool for that job is Redis — an in-memory store where a small read costs well under a millisecond. Memcached, the other common in-memory store, could hold the same data — but Redis comes with clustering and replication built in (replication meaning live copies on other machines), and this table has to survive its own machines failing; the last use case leans on exactly that. Now size it, by traffic rather than by bytes. Every online delivery does one lookup, so the directory serves about three million reads a second at peak. One Redis machine comfortably handles about a hundred thousand small operations a second. So we split the table into about thirty shards — a shard being one slice of a store that has been split across machines — with the device id hashed to pick the shard. The bytes, by contrast, are almost nothing: two hundred million entries at about a hundred bytes each is roughly twenty gigabytes across the whole cluster. Thirty machines for the read rate, a trivial amount of data — the directory is sized by throughput, not capacity.

The cost, said plainly: this is one more component that every single delivery depends on. If the directory is slow, chat is slow; if it is down, routing is down. We take that deal because the alternatives are worse, and because the dependency can be hardened — which is exactly what the last use case does.

The routing answer said 'forwards the message to chat server 340' as if that were nothing. How, exactly? The obvious way is a direct network connection from A's server to B's server. For one pair of servers, that is fine. But any of our four hundred servers may need to reach any other at any moment. Should every chat server hold a direct line to every other — or should there be one shared thing in the middle?

Work out what the direct mesh costs first, because for a small fleet it is genuinely the right call. With four hundred servers, each may need to talk to the other three hundred ninety-nine — about eighty thousand distinct links across the fleet. The sockets themselves are not the problem; we just saw one box hold half a million. The problem is the bookkeeping. Every server now tracks the health of every other server, reconnects to each one that drops, and rebalances every time the fleet grows. That workload grows with the square of the fleet size, and it lands on the exact tier whose CPU we are protecting. It works fine in the design review and turns into an operational swamp as the fleet grows.

So: one shared thing in the middle that every chat server talks to. Each server then holds one connection instead of three hundred ninety-nine, and nobody tracks anybody's health except the middle thing's. The pattern is called a message broker. That settles the shape — but not the product, and the products are not interchangeable.

Redis pub/sub is the lightest candidate, and Redis is already in our stack. But it is fire-and-forget: if a server is disconnected for even a moment, anything published to it in that gap is simply gone, and we would end up rebuilding delivery guarantees on top of it ourselves. RabbitMQ is a real broker with real guarantees, built around routing individual tasks to the programs that consume them — excellent up to tens of thousands of messages a second, which leaves it two orders of magnitude short of our three million. Kafka is different in kind: it treats messages as a stream you append to, and it is built to split that stream across machines, which is exactly the property our load demands. Kafka it is — and the answer below shows how it carries the load.

A shared message broker (Kafka). Every chat server holds exactly one connection — to Kafka. To forward a message, A's server publishes it, tagged for chat server 340. Chat server 340 is continuously reading its own stream, sees the message, and writes it down B's socket. All the all-to-all traffic still happens; only the wiring collapsed, from about eighty thousand managed links to four hundred — one per server.

How does Kafka itself carry three million messages a second? Partitions. A stream in Kafka is split into many partitions — independent slices, each living on some machine in the Kafka cluster — and we partition by the target chat server, so each chat server reads just its own slice. No single broker machine ever sees the whole flood, and adding chat servers just adds partitions, so the broker scales with the fleet. In machines: at about a kilobyte per message, three million a second is roughly three gigabytes flowing in and out per second, which a cluster of a few dozen broker machines carries comfortably. Each partition is also kept on disk for a short window, so a chat server that crashes and comes back resumes its stream from where it stopped — forwards that were in flight are not lost. And the broker must not become the new weak point: Kafka keeps copies of each partition on more than one machine in the cluster, so losing a broker machine loses nothing.

One thing the broker does not do: decide whether B is online. That is still the directory's one question. The broker only carries the forward once the directory has answered.

Time to look at the box we have been writing into. The day-one database has been quietly storing every message before its sender hears 'sent,' and it cannot keep doing it: a single node takes tens of thousands of writes a second, and peak is three million. So: what kind of store takes three million writes a second and still serves our one read — the recent messages in this chat — cheaply?

Look at how this system reads and writes, because the workload picks the store.

We only ever do one kind of read: open a chat and load its most recent messages. That is the whole read story. No joins, no searching across chats, no analytics on this path. The writes are the opposite story: constant, tiny, three million a second at peak, forever.

A relational database — Postgres, MySQL — is the default reach, and here it is the wrong fit twice over. It offers rich queries we would never run. And in exchange, a single node gives us tens of thousands of durable writes a second, when we need a hundred times that. Worse, a relational database does not spread one table across a hundred machines by itself — making it do that is a project you own forever. We would be paying for a query engine we never use while fighting its scaling from day one.

What about a log, like Kafka, since messages are already an ordered stream? A log is a superb transport — we just made it one. But as a store, it answers the wrong read. A log reads 'the whole stream, in order.' Our read is 'jump into one particular chat and give me its recent tail,' and you cannot jump into the middle of a log by chat cheaply.

There is a family of stores whose native shape is exactly our workload. It groups rows by a key and keeps them sorted inside each group. Writes are fast appends. Reading the recent tail of one group is cheap. And when you need more capacity, you add machines, and the store spreads itself across them — that part is built in. This family is called the wide-column store.

A wide-column store. Concretely: Cassandra, or its faster rewrite ScyllaDB — the store family Discord and many chat products run on. Messages are grouped by chat and kept in order inside each chat. Opening a conversation reads one group's recent tail. Writing is a fast append. Adding capacity is adding nodes. And the trade we sign is just as concrete: this store does the one read we designed for and nothing else. A question like 'every message this person sent last Tuesday, across all their chats' is not cheap here — if we ever need it, we build a separate index for it. We specialize, on purpose, because the workload is that lopsided.

The store is chosen; now it has to be split — the same sharding move the directory made, this time on the big table. One node takes about fifty thousand writes a second and peak is three million, so the store becomes about sixty shards, each on its own machine. One column of the row decides which shard each message lands on. Look at the table: which column would you split by, so that writes spread across all sixty shards and reading one chat stays cheap?

Take the three candidate columns one at a time, and test each against both jobs: spread the writes, keep the read on one shard.

Split by created_at, the timestamp? It sounds sensible — recent messages are the hot ones. But watch what happens to writes. Every message being written right now carries the same 'now,' so every single write lands on whichever shard owns the current time slice. One shard takes the entire three million a second while fifty-nine sit cold. And the read gains nothing, because we look messages up by chat, not by time. This is the trap answer: a true observation attached to the wrong mechanism.

Split by sender_id? Now the writes spread nicely — there are hundreds of millions of senders. It is the read that breaks. A conversation's messages come from many senders, so one chat's history is scattered across every member's shard, and opening a chat — the one read we optimized the whole store for — becomes a gather across the fleet. Hold onto the idea, though: a per-person split is exactly right for a different table, coming in the offline use case.

Split by chat_id? Every message in a conversation lands on the same shard, so opening a chat reads one shard. And with billions of mostly-small chats, writes spread across all sixty on average. The one weakness is honest and visible: a giant, hyperactive group is still one chat on one shard.

chat_id. Split by the key you read by: the store shards about sixty ways on chat_id. The giant-group weak spot from the debate is real, and it is a known hot spot with a planned fix — described under the hood below — rather than a surprise.

One check before moving on: 'sixty shards' answers only one of two different sizing questions. Sixty is the throughput answer — three million writes a second, over fifty thousand per node. Capacity is a separate and bigger answer. At about a kilobyte per message and a hundred billion messages a day, the store grows by roughly a hundred terabytes every day — even one year of that is tens of petabytes, which is why real chat products either cap how much history they keep or move old messages to cheaper storage. So disk and retention, not write rate, set the true machine count. Each shard is also replicated — about three copies — so a dead machine loses nothing. Put together, the real fleet is a couple hundred machines: sixty is the write floor; capacity and replication set the rest.

Two messages hit the same chat within the same millisecond. Which comes first? And when a phone reconnects after an hour away, how does it fetch exactly what it missed — no gaps, no duplicates? Both questions are really one question: what defines the order of messages in a chat?

The natural first answer is the timestamp — every message already carries one, so sort by it. But 'sort by time' quietly assumes two things that are false. It assumes no two messages share a time; at three million a second, two messages in the same chat land in the same millisecond all day long, and then there is a tie. And it assumes every server's clock agrees; they do not — real clocks drift apart by milliseconds. Neither error sounds dramatic, but our promise is that a reply never renders above the message it answers, and a few milliseconds of drift breaks exactly that.

The opposite extreme is a single global counter: every message in the whole system gets the next number, one perfect worldwide order. It works — and it costs everything, because every message on earth now queues at one counter for its number. That is a global bottleneck, bought for a guarantee nobody uses: no one ever reads all chats interleaved into one stream.

Between 'free but wrong' and 'perfect but ruinous' sits what the product actually promises: order matters within one chat. So put the coordination there, and only there.

Sortable id + per-chat sequence. Every message gets two numbers, and each does one job. The first is a sortable id — a plain sixty-four-bit number any server can mint on its own, with the time in its high bits, so ids sort roughly by time across the whole system with zero coordination. (The exact layout is in the figure below.) The second is the per-chat sequence: one, two, three within each conversation. This is the number that actually guarantees order.

So who hands out the sequence? Nothing new on the board. A chat's messages all land on one store shard — that was the point of splitting by chat — and each shard has a lead copy that accepts its writes. That lead copy is the chat's one writer: as it accepts each message (the same durable write that lets the sender hear 'sent'), it assigns the chat's next number from a counter in its memory. One writer per chat means the numbers come out gap-free, with no coordination beyond that one conversation — which is naturally one-at-a-time anyway. If that machine dies, another copy of the shard takes over, reads the highest sequence already stored for the chat, and continues from there. A lost counter costs one lookup, never a gap.

The sequence also answers the second question — catching up. 'Give me everything in this chat after the last sequence number I saw' is an exact request: no gaps, no duplicates, nothing to guess. That is the after_seq cursor in the catch-up API, and it is how a reconnecting phone fills in what it missed.

┌─────────────────────────────┬────────────┬─────────────┐
│ timestamp — 41 bits         │ machine id │ counter     │
│ (milliseconds; ~70 years)   │ 10 bits    │ 12 bits     │
└─────────────────────────────┴────────────┴─────────────┘
  high bits first → comparing two ids compares time first

  example, minted on machine 7, third id that millisecond:
  [ 1720950000123 ][ 0000000111 ][ 000000000011 ]

The directory lookup just came back empty: B has no live socket. The message still cannot be lost. So where does it wait until B's phone wakes — and how does the phone learn that it is waiting?

The tempting shortcut is the platform push service itself. Apple's APNs and Google's FCM exist to reach sleeping phones — so why not just send the message text through them and be done? Because they are someone else's system. They rate-limit you, they may merge several notifications into one or quietly drop some under pressure, and they hold none of our promises: no ordering, no durability, no catch-up. A push that carries the message is a message whose delivery you can no longer vouch for. So the push gets a smaller job: waking the phone. It is good at that.

The other shortcut fails faster: leave the message parked on a chat server until B comes back. But a chat server holds live sockets and it can die — and anything parked on it dies too. We already have a place whose whole job is keeping messages safe; the chat-server tier is not it.

So the job splits in two. Holding the message durably is our side: a mailbox in the store. Waking the phone is the platform's side: a push with nothing inside it.

Offline inbox + push. Two pieces, each doing the half it is good at. A durable offline inbox: a per-recipient mailbox that A's chat server appends the message to, so it survives — for hours or for weeks — until B returns. And a push notification, handed to the platform's service (APNs on iOS, FCM on Android), that carries no message content at all. It only tells the sleeping phone 'you have messages,' so the phone knows to reconnect and pull them from us.

Step by step, the same way we walked the send: (1) A's chat server looks B up in the directory and gets 'no live socket.' (2) It appends the message to B's inbox — a durable write, so nothing can be lost from here. (3) It hands a wake-up to APNs or FCM for B's device. (4) B's phone wakes, opens its WebSocket to a chat server, and asks for its inbox. (5) The chat server reads B's inbox in arrival order and streams the messages down the socket — each one carrying its per-chat sequence, so the phone drops it into the right conversation at the right position. (6) B's phone acknowledges what it received, and those inbox entries are cleared. From there, B is back on the live path.

What store is the inbox, concretely? No new technology: it is a second table on the same Cassandra-class store as messages, just keyed differently. Messages are keyed by chat; the inbox is keyed by device — whose mailbox it is. Each row carries the chat id, the message's per-chat sequence, and the body. Reading a mailbox is then one shard scan — every row for this device, in arrival order — exactly the shape of read this store is good at. Its write load is real, and the store's sizing already speaks its language: a message to offline recipients costs one inbox append per absent device, and you scale for those appends the same way as the messages table — by adding machines. The other costs we accept: one more table, and a dependency on Apple's and Google's push services for the wake-up. A lost push costs only latency, never the message — the phone also checks its inbox every time the app opens.

This deep dive opens with the design already broken. A sent to a thousand-member group; the send path is trying to deliver to all thousand members before it answers A, and A's app is timing out. What change gets A a fast 'sent' while a thousand deliveries still happen?

Name the two promises hiding inside the word 'send,' because they cost wildly different amounts. Promise one: your message is safe. That is what A is actually waiting to hear, and it costs one durable write. Promise two: everyone has it. That costs up to a thousand deliveries. The naive path welds the two together, so A waits on the expensive one.

Could we just add more chat servers? This is the trap answer, because it scales the symptom. Twice the servers still run the same thousand-delivery loop inside every group send — the sender still waits on the biggest group, just on a newer machine. The problem is not capacity. It is that the thousand deliveries sit on the send path at all.

Could we keep delivering inline, for correctness? For a five-person chat — honestly, yes. Many systems fan out small groups inline and only queue the big ones. It just cannot survive the tail, and the tail is in our requirements: a thousand members.

So separate the promises. Acknowledge A the moment the message is stored. Move the thousand deliveries off A's path entirely, onto something that can chew through them at its own pace.

Async workers off the broker. The sender's chat server does two cheap things and then answers. It stores the message — that is the promise A is actually waiting on. And it drops one small job onto the Kafka broker we already have: 'deliver this to group G.' Then it tells A 'sent.' Total cost on A's path: one durable write and one hand-off to the broker. Behind that answer, a pool of fan-out workers pulls the job, reads group G's member list from a small group-membership table (each group's list of member ids — read once per group send), and delivers to each member at its own pace: online members through the broker to their chat servers, offline members into their inboxes with a push. Nothing here is a new invention — it is the broker from the routing use case, plus a worker pool sized by delivery rate: scale it like any pool of interchangeable workers, by adding more when the backlog grows.

The trade this buys is worth saying plainly: group delivery becomes eventually consistent. A is told 'sent' while the last members have not received it yet, so for a moment some of the group has the message and some does not. For chat, that is the right trade — a small delivery lag inside a big group is invisible, and a multi-second freeze on every group send is not.

  A ── send ──▶ chat server ── 1 · store ──▶ message store
                    │
                    ├── 2 · one job on the broker: 'deliver to group G'
                    │
                    └── 3 · ack A  ✓ (~200 ms — A is done here)

  broker ──▶ fan-out workers ── read members of G (1,000) ──┐
                                                           │
                online member  ──▶ broker ──▶ their chat server ──▶ socket
                offline member ──▶ inbox + push

When B's phone receives A's message, A's bubble should tick to 'delivered'; when B opens the chat, to 'read.' How do those ticks travel back to A — and what stops a thousand-member group from burying A under a thousand of them?

Strip the mystery first: a receipt is itself a message. 'That message you sent has been read' is a tiny piece of text addressed to A — and we have just spent five use cases building a machine whose whole job is getting a tiny piece of text to a person, wherever they are, online or not. Building a separate receipts subsystem would mean building that machine twice. So receipts ride the machine we have, pointed the other way.

The one genuinely new problem is volume, and it comes from groups. Send one message to a thousand-member group and up to a thousand 'delivered' receipts and a thousand 'read' receipts come back — a thousand times as many acknowledgements as the message they describe. Sending each one back as its own message is perfectly right for a 1:1 chat — one tick each way — and unsurvivable for a big group. So the group path gets one extra move: batch the flood before it reaches the sender.

Send receipts back like messages, batched for groups. For a 1:1 chat, a receipt simply rides the machine backward. When B's phone receives or reads A's message, B's chat server sends a small receipt addressed to A — and it routes exactly like any message: look A up in the directory, forward through the broker, push down A's socket, and A's bubble ticks. If A is offline, the receipt waits in A's inbox like anything else. For a group, one extra move: the receipts from a thousand members flow back through the fan-out workers, and the workers batch them — one 'delivered to thirty people' update to A's phone instead of thirty separate messages.

One product question settles the storage: does a group chat show a count ('seen by twelve') or the list of who ('seen by Alice, Bob…')? Both travel the same way; they differ only in what you keep. A count is one number you bump as receipts arrive. The list keeps one row per reader — more storage, but you can show names. Pick per product; this design defaults to ticks with batched group counts.

A chat server is stateful: it holds half a million specific people's live sockets, and those people's entries in the directory point at it. So when one dies, two things happen at once — half a million connections drop, and half a million phones notice within seconds of each other. What keeps that from becoming an outage?

Start with what actually happens in the second after a chat server dies, because the obvious safety argument misses it. The argument says: we have four hundred of them, losing one is a quarter of a percent, the fleet absorbs it. That works for stateless servers, where any survivor can serve any request. Our tier is not that. The dead box held half a million specific people's sockets, and no survivor has them.

So all half a million phones notice their dead connection within seconds of each other — and reconnect at the same moment. That reconnect stampede is the real danger: a wall of simultaneous reconnections landing on the balancer, the surviving servers, and the directory all at once, hard enough to knock over machines that were perfectly healthy. And a second problem hides behind the first: until each phone reconnects, its directory entry still points at the dead machine, so messages for those users are being forwarded to a box that cannot deliver them.

Redundancy did nothing about either problem. What survives the death is making the reconnection itself survivable, and making the one thing every delivery depends on — the directory — able to lose its own machines.

Replicate the directory + spread the reconnects out. Two moves, one for each half of the problem. First, the directory must have no single point of failure, because every delivery leans on it. It is already split into about thirty shards; now each shard gets replicas — copies on other machines — so losing one directory machine costs only a moment for one shard's slice, and even that slice heals itself as its users reconnect and rewrite their entries. Second, spread the reconnections out in time. Instead of every dropped phone retrying the instant it notices, each phone waits a short random delay first — so half a million reconnections land smeared across several seconds instead of in one spike. That deliberate randomness has a name: jitter. Together, the two moves turn a chat server death into a brief blip: dropped people are offline for a second or two, reconnect on healthy servers, and the directory follows them there.

  no jitter — everyone retries the instant they notice:

  load │█
       │█
       │█▁▁▁▁▁▁▁▁▁▁▁
       └────────────── time

  jittered — each phone waits a short random delay first:

  load │
       │▂▄▅▅▄▃▂▁
       │▁▁▁▁▁▁▁▁▁▁▁▁▁
       └────────────── time (a few seconds)