The problem: the server can't start the conversation#
TL;DRthe 30-second version
- HTTP is request-response: the server can't say anything until a client asks. Polling ('anything new?' on a loop) wastes requests and still lags.
- A WebSocket starts as a normal HTTP request with an Upgrade header. The server answers 101 Switching Protocols, and the same TCP connection becomes a permanent two-way pipe.
- After that, both sides send messages whenever they want, as small framed chunks, with no new request per message.
- A ping/pong heartbeat spots a connection that died silently, so the server can drop it and free what it was holding.
- The catch is scale: every open connection holds memory and a file descriptor on the server, so a million of them needs a fleet of connection servers plus a pub/sub backplane to route each message to whichever server holds the recipient.
Start with what HTTP actually does. Bob's browser sends a request, the server sends back a response, and the exchange is over. That's the whole shape: the client asks, the server answers, done. It works beautifully for loading a page. But notice what the server can never do — it can't reach out to Bob on its own. When Alice's "lunch?" lands on the server, there's no open line to Bob waiting for it. The server is holding a message with no way to deliver it.
The obvious fix is to have Bob's browser keep asking. Every two seconds it sends a request: "any new messages?" This is short polling, and it has two problems you can put numbers on. First, latency: a message can sit up to two seconds before the next poll picks it up, so the chat feels sluggish. Second, waste: if 10,000 people have the chat open and each polls every two seconds, that's 5,000 requests a second, and almost every one comes back empty. You're paying for a constant stream of "nope, nothing" just in case something arrives.
You can sharpen it with long polling: the browser sends a request and the server holds it open, not answering until there's actually something to send. That cuts the latency and the empty replies. But it's still request-response underneath — the server answers with one message, the connection closes, and the browser has to fire a brand-new request to wait for the next one. On a busy channel that's a constant churn of reconnecting. The real question sharpens too: instead of faking a push with clever request timing, can we keep one connection open and let both sides send on it whenever they want?
Build it: reuse the connection you already have#
Here's the move. When Bob's browser made that HTTP request, a TCP connection was opened to carry it. TCP is a reliable, ordered stream of bytes (the transport under almost every request — see /tcp/sim), and it is already two-way: bytes can flow in both directions at once. The only thing stopping the server from pushing to Bob is the HTTP rule sitting on top of TCP, which says one request gets one response and then you're done. So don't tear the connection down. Ask to change the rules on it.
The browser sends an ordinary HTTP request — a plain GET, the same kind that starts any page load (see /http/sim) — but it adds one header: "Upgrade: websocket". That header is a question: can we switch this connection to a different protocol? If the server understands WebSockets and agrees, it replies not with the usual 200 and a page, but with the status code 101 Switching Protocols. From that instant, the HTTP request-response rule is gone. The raw TCP connection underneath stays open, and both ends are now free to send on it whenever they like.
That open, two-way connection is a WebSocket. "Two-way at the same time" has a name — full-duplex — meaning both ends can send at once without taking turns, the way a phone call lets both people talk over each other, unlike a walkie-talkie where one side speaks and then the other. One handshake at the start, and then a single long-lived connection carries messages in both directions for as long as it stays open.
Now both sides can send bytes, but there's a snag: a TCP connection is just a stream of bytes with no natural edges. If the server sends "lunch?" and then "where?", the bytes arrive back to back and Bob's browser can't tell where one message stops and the next begins. So WebSocket messages travel as frames. A frame is a small header that says "this is a text message, N bytes long," followed by exactly those N bytes. The receiver reads the header, collects exactly N bytes, and hands up one clean message. Framing is what turns a raw byte stream back into discrete messages.
One last piece. A WebSocket can stay open for hours, and over that long a connection can die without anyone being told — Bob's phone drops off wifi, or a network box somewhere in the middle quietly forgets the connection. The bytes just stop, with no error. So the protocol has a built-in check: one side sends a tiny ping frame, and the other must reply with a pong frame. This is the heartbeat. If a few pings go unanswered, you know the connection is dead — so you close it and reclaim everything it was holding. The heartbeat also keeps a truly idle connection from being closed by an impatient proxy in the middle, because a ping every so often looks like activity.
Go deeperUnder the hood: the frame format and why the client masks its data
A frame's header is tiny. The first byte holds an opcode saying what kind of frame it is — text, binary, close, ping, or pong — plus a bit marking the final frame of a message. The next bits give the payload length. Then comes the payload. Small messages carry only a couple of bytes of overhead, which is a big reason WebSockets are cheaper per message than re-sending HTTP headers on every poll.
One rule looks odd until you see the reason: every frame the client sends to the server must be masked, meaning its payload is scrambled by XOR against a random 32-bit key that the client includes in the frame. This isn't for secrecy (use wss://, WebSocket over TLS, for that). It's to protect old network equipment in the middle. Some proxies and caches were built long before WebSockets and try to interpret bytes flowing past as HTTP. A malicious web page could otherwise craft a WebSocket payload that looks like a fake HTTP request and trick such a proxy into caching a poisoned response. Masking makes the client's bytes unpredictable, so they can't be shaped into that attack. The server does not mask its frames, and it must close the connection if a client ever sends an unmasked one.
The scaling reality: a million open connections#
A WebSocket's whole point — staying open — is also what makes it expensive. A normal HTTP request is over in milliseconds and the server moves on. An open WebSocket, even one where nobody is typing, keeps costing the server the entire time it's connected. Three things add up per connection: a file descriptor (a small integer the operating system hands out for every open connection, and there is a hard limit on how many one machine can have), the kernel's send and receive buffers for that socket, and whatever your app keeps in memory per user — who they are, which chat rooms they're in. Call it tens of kilobytes all told for an idle connection.
PredictYour chat app has 1,000,000 users connected at once. Say each idle WebSocket costs about 40 KB of server memory in socket buffers and per-user state. How much memory is that, and what does it tell you about running it on one machine?
Hint: Multiply connections × bytes-per-connection. Then remember the file-descriptor limit is separate from memory.
1,000,000 × 40 KB ≈ 40 GB — just to hold the connections idle, before a single message is processed and before your app logic gets any memory. And that's only the memory side: each connection also burns a file descriptor, and a single process tops out at a fixed number of them (tuned into the tens or hundreds of thousands, not millions, without heroic kernel tuning). So one machine can't hold a million connections comfortably. You spread them across a fleet of machines whose one job is to hold connections open. WhatsApp famously tuned FreeBSD to hold 2 million+ connections on a single server, but that took deep OS-level work — the ordinary answer is: use many servers.
Spreading connections across many servers creates a new problem, and it's the heart of the whole design. Alice is connected to server A. Bob is connected to server B. Alice sends "lunch?", so it lands on server A — but the only open connection that can reach Bob lives on server B. Server A has no line to Bob at all. Delivering the message means getting it from the server that received it to the server that holds the recipient.
The fix is to put a shared message bus behind the connection servers. When Alice's message arrives at server A, server A doesn't try to find Bob itself. It publishes the message to a pub/sub backplane — a message bus that every connection server subscribes to (this is exactly the publish/subscribe fan-out at /pubsub/sim). The server holding Bob is subscribed, picks the message up, and pushes it straight down Bob's open socket. The connection servers stay dumb: they hold sockets and relay to and from the bus. The routing is the bus's job.
That's the shape of every large real-time system: a tier of connection servers that do nothing but hold WebSockets open, a pub/sub backplane that fans each message to the right server, and a registry that tracks which user is on which server. The chat logic — history, groups, delivery receipts — sits behind that tier, and this page is only about the transport layer under it.
Short polling vs long polling vs SSE vs WebSockets
WebSockets aren't the only way to get fresher data than plain request-response. There are four common options, and the honest comparison turns on two questions: how quickly does new data reach the client, and can the client send back on the same channel? One option that needs a gloss is SSE — Server-Sent Events — a one-way stream from server to browser over a single long-lived HTTP response, with the browser automatically reconnecting if it drops. Great for a feed of updates, but the client can't send anything back on it.
| Approach | How it works | Direction | Latency | Good for |
|---|---|---|---|---|
| Short polling | Client re-asks on a fixed timer | Client asks, server answers | Up to the poll interval | Rarely-changing data, simplest possible |
| Long polling | Server holds the request until there's data | Client asks, server answers | Near-instant one way | Occasional pushes, no WebSocket support |
| SSE | One long HTTP response streams events down | Server → client only | Near-instant, server → client | Live feeds, notifications, tickers |
| WebSocket | Upgraded connection, framed messages | Both ways, full-duplex | Near-instant, both ways | Chat, multiplayer, live trading |
Read the table top to bottom and it's a ladder: each rung either lowers latency or adds a direction, and WebSockets sit at the end because they give you both. But further down the ladder is not automatically better — SSE rides plain HTTP and is simpler to run, and it's the right pick whenever data only flows one way.
When not to reach for WebSockets
A WebSocket is a persistent, stateful connection your servers must hold open and account for. That's real weight, so only take it on when the workload actually needs two-way, low-latency pushing.
- Data flows one way (a stock ticker, a live sports score, notifications, a progress feed): use SSE. It's one-way, rides ordinary HTTP, and reconnects on its own — simpler to build and to scale than a WebSocket.
- Updates are occasional and the client mostly initiates (a dashboard that refreshes every minute): plain HTTP or long polling is fine. A persistent connection buys you nothing here.
- Both sides push frequently and latency matters (chat, multiplayer games, collaborative editing, a trading screen that shows live prices and also sends orders): this is exactly what WebSockets are for.
Failure modes & gotchas
Why do WebSockets need sticky sessions at the load balancer?
A WebSocket is a single connection pinned to one server for its whole life. If a load balancer spread the bytes of that connection across several servers, only one of which did the handshake, it would break. So the balancer must keep each connection glued to the server that answered its upgrade — that's a sticky session. It also means a reconnect might land on a different server, which is fine precisely because the per-user routing state lives in the backplane and registry, not on any one connection server. Balancing long-lived connections is its own puzzle — see /loadbalancer/sim.
What happens if the server sends faster than a client can read?
The server's send buffer for that socket fills up, because the slow client isn't draining it. This is backpressure, and ignoring it is how a server runs out of memory: buffers for thousands of slow clients pile up. The server has to react — drop messages it can afford to lose, coalesce updates (send only the latest position, not every one), or disconnect a client that's fallen hopelessly behind. See /backpressure/sim for the general pattern.
HTTP/2 already lets many requests share one connection — doesn't that replace WebSockets?
No. HTTP/2 multiplexes many requests over one connection (see /http2-3/sim), which is great, but it's still fundamentally request-response: the client asks and the server answers. It has no clean, general way for the server to push an unprompted message to the client. WebSockets give you a true two-way channel where either side speaks first. They solve different problems — multiplexing many requests versus pushing arbitrary messages both ways.
Do WebSockets work through corporate proxies and firewalls?
Mostly, and the reason is the handshake is plain HTTP on the normal web ports. The safest setup is wss:// — WebSocket over TLS — because an encrypted connection is passed straight through by middleboxes that would otherwise try to inspect and mangle it. Heartbeats matter here too: an idle connection can be silently closed by a proxy, and a periodic ping keeps it looking alive.
How does the client find out the connection dropped, and what should it do?
The heartbeat catches a silent death — miss a few pongs and the connection is known dead. The standard client response is to reconnect with backoff (wait a little, then a little longer, so a server coming back up isn't stampeded), re-run the handshake, and re-subscribe to whatever it was watching. Because connection state is disposable, reconnecting is safe: the durable data lives behind the connection tier, not in the socket.
QuizYou're building a live sports scoreboard: thousands of viewers watch scores update, and data only ever flows from the server to the viewer. Which transport is the best default, and why?
- WebSockets — they're the fastest, so always use them for anything real-time.
- SSE — the data is one-way (server → client), so a simpler one-way stream over plain HTTP with auto-reconnect fits, without the cost of full-duplex per-connection state.
- Short polling — it's the simplest, and latency never matters for scores.
- HTTP/2 server push — it's designed exactly for streaming live updates.
Show answer
SSE — the data is one-way (server → client), so a simpler one-way stream over plain HTTP with auto-reconnect fits, without the cost of full-duplex per-connection state. — The data only flows one way, from server to viewer, so Server-Sent Events are the natural fit: a single long-lived HTTP response streams each score update down, the browser reconnects automatically, and you avoid the full-duplex plumbing and per-connection write state a WebSocket carries. WebSockets would work but are overkill for a one-way feed. Short polling adds latency and wasted requests. HTTP/2 server push was designed to pre-send page assets, not to stream application events, and is deprecated in practice.
In the wild
- Slack keeps a WebSocket open per client for its real-time layer — new messages, typing indicators, presence, and reactions all arrive as pushed events rather than being polled for, which is what makes the app feel live.
- WhatsApp Web mirrors your phone over a persistent connection so a message shows up on the laptop the instant it arrives. WhatsApp's backend is the canonical example of connection scale: it tuned FreeBSD to hold over 2 million connections on a single server, running each connection as a lightweight Erlang process.
- Trading and crypto dashboards push live prices down a WebSocket and accept orders back up the same connection — a genuinely two-way, latency-critical case where SSE's one-way limit wouldn't do.
- Figma and Google Docs use persistent connections for multiplayer editing: every cursor move and edit is pushed to the other people in the document in real time, which only works with a low-latency two-way channel.
In an interview
"How does chat push a message instantly, and how would you scale it to a million users?" is a staple, and it rewards separating the protocol from the scaling. Lead with the protocol in one breath, then spend most of your time on the fan-out, because that's the part that shows systems thinking.
- State the problem first: HTTP is request-response, so the server can't push; polling wastes requests and adds latency. That motivates everything.
- Walk the handshake: an HTTP GET with an Upgrade header, a 101 Switching Protocols reply, and the same TCP connection becomes a full-duplex WebSocket carrying framed messages.
- Name the per-connection cost out loud: a file descriptor plus socket buffers plus per-user state, tens of KB each — so a million connections is tens of GB and many machines, not one.
- Draw the fan-out: a tier of connection servers holding sockets, a pub/sub backplane routing each message to the server that holds the recipient, and a registry mapping user → server. This is the answer they're actually digging for.
- Name the operational details that signal depth: heartbeats to detect dead connections, sticky sessions to pin a connection to its server, backpressure when a client reads slowly, and reconnect-with-backoff on drop.
PredictThe interviewer asks: "A user's message lands on the server they're connected to, but the recipient is connected to a different server entirely. How does it get delivered?" What's the strong answer?
Hint: The receiving server can't see the recipient's socket. What sits between all the connection servers?
Don't try to make the receiving server find the recipient directly — it has no connection to them. Instead, the connection servers sit behind a shared pub/sub backplane. The server that received the message publishes it to the bus; the server holding the recipient's socket is subscribed, picks it up, and pushes it down that socket. To know which server holds the recipient, keep a registry mapping each user to their current connection server (updated on connect and disconnect), or have every server subscribe to the recipient's channel so only the holder delivers. The one-line version: connection servers are dumb socket-holders, and a pub/sub backplane does the cross-server routing. That decoupling is also what makes reconnecting to any server safe.
References
- RFC 6455 — The WebSocket Protocol — The spec: the upgrade handshake, frame format, masking, and ping/pong.
- MDN — The WebSocket API — How to open and use a WebSocket from a browser.
- MDN — Writing WebSocket servers — The handshake, the Sec-WebSocket-Accept computation, and framing on the server side.
- WHATWG HTML — Server-Sent Events — The SSE spec: the one-way EventSource stream and auto-reconnect.
- MDN — Using server-sent events — SSE from the client side, and when it beats a WebSocket.
- High Scalability — How WhatsApp scaled connections — The real numbers behind millions of connections per server.