Ch. 13 · System Design

System Design: Presence Tracking

Build online indicators for 10 million users: heartbeats with TTLs, why the TTL must cover lost beats, and fan-out only to who's watching.

~7 min readadvancedupdated Nov 1, 2026

The green dot next to a name in Slack, WhatsApp or a multiplayer game lobby looks like the simplest feature in the product. Behind it is a question distributed systems are bad at answering: is that other machine still there? A phone that loses signal in a tunnel doesn’t announce it. A laptop that sleeps doesn’t close its WebSocket politely. A presence system has to infer “offline” from silence, and then tell the people who care without drowning the system in updates.

That makes it a compact design exercise with three parts that pull against each other: state that expires with time, a steady heartbeat write load, and the fan-out of every status change to the people watching.

Online is something you infer, not something you store

The tempting design is a status column: set it to online when a user connects, offline when they disconnect. It breaks the first time a client disappears without disconnecting. The row says online forever, or until some cleanup job notices.

Flip it around. Each connected client sends a heartbeat every 30 seconds, and each heartbeat refreshes a key with an expiry:

SET presence:u42 gw-17 EX 65
Text

A user is online if presence:u42 exists. When the client goes silent, nothing has to happen: the key expires on its own. The value records which gateway server holds the connection, which is useful for routing messages to that user later. A clean disconnect (the user logs out, or the socket closes with a proper close frame) can delete the key immediately, so the common case is instant and the crash case is bounded by the TTL.

“Last seen 5 minutes ago” is a separate piece of data. Write a last_seen_at timestamp to the user’s profile store when the presence key is removed or expires, or at most once every few minutes while online. It doesn’t need the write rate of heartbeats.

Choosing the heartbeat and the TTL

These two numbers set your write load and your error rate.

The write load first. With 10 million concurrent users, a 30-second interval, and an assumed four online/offline changes per user per hour, a quick calculation in Node 22:

const users = 10e6, interval = 30;
console.log('heartbeat writes:', Math.round(users / interval).toLocaleString('en-US') + '/s');
const changes = users * 4 / 3600;
console.log('status changes:', Math.round(changes).toLocaleString('en-US') + '/s, naive fan-out to 200 friends:',
  Math.round(changes * 200).toLocaleString('en-US'), 'msgs/s');
JavaScript
heartbeat writes: 333,333/s
status changes: 11,111/s, naive fan-out to 200 friends: 2,222,222 msgs/s
Text

A third of a million small writes a second is well within reach of a Redis cluster sharded by user id, but it’s not free, and halving the interval doubles it. That’s why heartbeats are usually 20 to 60 seconds, not 2. The second line, the fan-out, comes back in a later section.

The TTL is the easier number to get wrong. Setting it just above the interval (say 35 seconds for a 30-second heartbeat) means one lost or late heartbeat flips the user offline, and then online again 30 seconds later. Here’s a simulation of 20,000 users who stay online for a full hour, with 30-second heartbeats, up to 3 seconds of network jitter, and a chance of each heartbeat being lost:

let seed = 7; const rand = () => (seed = (seed * 16807) % 2147483647) / 2147483647;
const interval = 30;

function flapsPerUserHour(ttl, loss, users = 20000) {
  let flaps = 0;
  for (let u = 0; u < users; u++) {
    let lastSeen = 0;
    for (let t = interval; t <= 3600; t += interval) {
      const arrive = t + rand() * 3;
      if (rand() < loss) continue;              // heartbeat lost
      if (arrive - lastSeen > ttl) flaps++;     // key expired before this one arrived
      lastSeen = arrive;
    }
  }
  return (flaps / users).toFixed(3);
}
for (const loss of [0.01, 0.05]) for (const ttl of [35, 65, 95])
  console.log(`loss ${loss * 100}% ttl ${ttl}s -> false offline flaps per user-hour: ${flapsPerUserHour(ttl, loss)}`);
JavaScript
loss 1% ttl 35s -> false offline flaps per user-hour: 1.182
loss 1% ttl 65s -> false offline flaps per user-hour: 0.012
loss 1% ttl 95s -> false offline flaps per user-hour: 0.000
loss 5% ttl 35s -> false offline flaps per user-hour: 5.657
loss 5% ttl 65s -> false offline flaps per user-hour: 0.277
loss 5% ttl 95s -> false offline flaps per user-hour: 0.015
Text

With a 35-second TTL and only 1% loss, a user who never left appears to go offline more than once an hour. On a flaky mobile network at 5% loss, it’s five or six times an hour, and every one of those flaps generates two status-change messages to everyone watching. Raising the TTL to just over two intervals (65 seconds) cuts false flaps by about a factor of a hundred, because now two consecutive heartbeats have to be lost. The cost is that a genuine crash takes up to 65 seconds to show. For a chat app that’s fine; for a game matchmaker that needs to know within seconds, you’d shorten the interval rather than tighten the TTL.

Fan-out: tell only the people looking

The second number from the estimate is the scary one. If users change state about 11,000 times a second and each has 200 contacts, pushing every change to every contact is over two million messages a second, and most of them go to people who aren’t looking at that contact anyway. Large groups make it worse: a 10,000-member Slack channel would broadcast every member’s every flap to every other member.

The fix is to send presence only to clients that are currently displaying the user:

  • When a client opens a conversation list or a chat, it subscribes to presence for the user ids visible on screen, maybe 20 of them, and unsubscribes as they scroll off.
  • The presence service keeps a reverse index, watchers:u42 -> {gw-3, gw-17}, by gateway server rather than by individual client, so a status change produces one message per gateway, which then fans out to its local sockets.
  • On subscribe, the client gets the current state in one batched read (MGET over 20 keys), then receives only changes.
  • For huge groups, show a count (“1,204 online”) computed periodically instead of individual dots.

Two smaller tricks cut the remaining traffic. Batch updates per gateway every second or two instead of sending each change immediately. And add a grace period before announcing “offline”: when a socket closes uncleanly, wait 10 or 15 seconds before broadcasting, because mobile clients often reconnect within that window. Combined with a sensible TTL, this removes most of the visible flicker.

Multiple devices and other edges

A user with a phone and a laptop is online if either is connected. A single key per user gets this wrong: the laptop closing would delete the key while the phone is still connected. Use one entry per session instead, for example a Redis hash presence:u42 with one field per session id, expired per field with HEXPIRE (Redis 7.4 and later) or tracked in a sorted set scored by last heartbeat, and compute “online” as “any live session”. The same structure gives you richer states like “active on mobile”.

“Away” or “idle” is client-reported: the app notices no input for 10 minutes and says so in its next heartbeat. The server can’t infer it.

Privacy settings (“hide my status”) are applied at read time in the presence service, not in the client, or anyone with a modified client can see through them.

Interview question: the flickering contact list

Users complain their contacts’ green dots blink on and off every minute or so, mostly on mobile. The system uses 30-second heartbeats and a 40-second Redis TTL. Presence changes are pushed to all of a user’s contacts over WebSockets. What’s going on, and what would you change?

Answer

A 40-second TTL leaves only 10 seconds of slack. Any heartbeat lost or delayed past that, which is common on mobile networks and when phones throttle background apps, makes the key expire, so the contact goes offline and comes back with the next heartbeat. Rerunning the simulation above with a 40-second TTL gives 1.17 false offlines per user-hour at 1% loss and 5.65 at 5%, essentially the same as 35 seconds, because a single lost heartbeat always opens a gap of at least 60 seconds. Because every change is pushed to every contact, each false flap is visible to dozens of people, which is why the complaints are widespread.

Change three things. Raise the TTL to about 2.5 intervals (75 seconds), so two lost heartbeats in a row are needed to flip state. Add a grace period of 10 to 15 seconds after an unclean disconnect before announcing offline, so quick reconnects never broadcast. And stop pushing to all contacts: only clients that have subscribed to a user, because that user is visible on their screen, receive updates, and gateways batch them. Then measure: track false-offline events (offline followed by online within two minutes for the same session) as a metric before and after the change.

Further reading

More in System Design

read ✓System Design · hard

System Design: Bloom Filters

Size a Bloom filter from item count and target error rate, see why it never gives false negatives, and know when it's the wrong tool.

~8 min readread →
read ✓System Design · hard

System Design: Time-Series Metrics

Design a metrics store: count series before samples, keep labels bounded, roll up with retention tiers, and never average percentiles.

~7 min readread →
read ✓System Design · mid

System Design: URL Shortener

Size a URL shortener from 100M links a month, pick between counter-based and random codes, and keep a 4,000 req/s redirect path cheap.

~7 min readread →
esc