Realtime Systems WebSockets Redis TTL

How Teams Knows You're 'Active' in Real Time

One tiny green dot. A surprisingly complex distributed system — heartbeats every 5 seconds, Redis keys expiring in 30, pub/sub fanout to thousands of teammates, and multi-device aggregation you never notice.

9 min read Heartbeats · Redis TTL · Pub/Sub Fanout Episode 02

One tiny green dot hiding a distributed system

You open Slack. Your colleague's avatar shows a green dot — "Active." You send a message. Within milliseconds, they see a notification. Ten minutes later they step away — the dot turns grey. No page refresh. No button click. The whole system just knows.

That green dot feels trivial. But underneath it, Slack has to answer a deceptively hard question: is this person actually at their computer right now? — for millions of users, across multiple devices, through dropped connections and spotty mobile networks.

The core challenge: Presence state is ephemeral — it lives for seconds, not years. It can't wait for HTTP polling. It has to propagate to every teammate across servers that have never talked to each other. And when someone's laptop dies, the system needs to figure that out on its own — with no goodbye message.

Why presence is one of the hardest things to scale

Active/Away status isn't just a notification — it's a continuous presence signal. And presence systems combine the worst of two worlds: they're as write-heavy as a logging system and as read-heavy as a CDN.

Four constraints working against you simultaneously

Scale: Slack serves tens of millions of daily active users. Each user's presence state may need to be visible to hundreds of teammates. A naive fan-out produces billions of presence updates per minute across the system.

Ephemerality: Presence isn't permanent. If a user closes their laptop without logging out — no explicit "I'm going away" event fires. The system must detect absence through silence, not through a message.

Multi-device: The same user might be active on their MacBook, iPhone, and iPad simultaneously. Each device has its own connection. "Is this user active?" requires aggregating across all of them — and correctly handling the case where the laptop goes away but the phone stays active.

Network instability: Mobile connections drop constantly. A dropped WebSocket connection doesn't mean the user is gone — they might just be going through a tunnel. The system needs to distinguish "network blip" from "actually away."

Scale reality: At 10M concurrent users, each sending a heartbeat every 5 seconds, that's 2 million presence writes per second — before any fan-out to subscribers. This is why presence has its own dedicated infrastructure tier in every major chat platform.

The system that makes the green dot work

Slack's presence runs on a Heartbeat + WebSocket + Redis TTL + Pub/Sub stack. The key insight: instead of the client saying "I'm away" when they leave, it continuously says "I'm still here" — and silence means gone. Click any node to inspect it.

presence-architecture.flow
Click any node to inspect · Heartbeat packets flow every ~5s

How each component works

Heartbeats — the pulse of presence

The core mechanism is the heartbeat: your Slack client sends a small PING frame over its WebSocket connection every ~5 seconds while the app is in focus and the user is active. This isn't a custom protocol — WebSocket has native PING/PONG frames built into RFC 6455 for exactly this purpose.

The presence service receives this ping, refreshes a Redis key with a fresh TTL, and the user stays "Active." Stop sending pings — key expires — user goes "Away." Simple. Elegant. Works at any scale.

WebSocket — the persistent channel

Slack keeps a persistent WebSocket connection between your client and the nearest edge server. This connection serves double duty: it carries heartbeats inbound (your presence) and pushes all real-time events outbound (messages, reactions, other users' presence changes).

Unlike HTTP, a WebSocket connection has a measurable liveness property: if the TCP connection drops, the server knows immediately. This gives the presence service a second signal — WebSocket disconnect — in addition to TTL expiry.

Redis TTL — the expiry engine

When a heartbeat arrives at the presence service, it executes: SET presence:{userId}:{deviceId} "active" EX 30. Thirty seconds is deliberately generous — it smooths over network blips while still catching genuine inactivity within a human-perceptible timeframe.

Redis's keyspace notification feature fires an event when a key expires. The presence service subscribes to these expiry events and translates them into "user went Away" signals — no polling required.

Pub/Sub fanout — propagating the green dot

When a presence state changes — active, away, or DND — the presence service publishes to a Redis channel keyed by workspace: presence:workspace:{workspaceId}. Every gateway serving members of that workspace is subscribed and pushes the update down open WebSockets.

For large workspaces (tens of thousands of members), this naive fanout is too expensive. Slack uses subscription-based filtering: clients only subscribe to presence changes for users they can currently see in their UI — open DMs, visible channels, active sidebar conversations.

Multi-device aggregation

The presence service tracks each device separately: presence:{userId}:{deviceId}. The aggregated status for a user is derived by a simple rule: active on any device = active overall. The service maintains a second key, presence:agg:{userId}, updated whenever any device key changes.

This means when you lock your MacBook but stay active on your phone, your teammates still see green — correctly.

What breaks — and how it recovers

Click a failure to simulate how the system responds:

— Select a failure scenario

Network drop — the quiet failure

When a user's WiFi drops, their WebSocket connection dies silently. The gateway detects the TCP close and immediately marks the connection gone — but the presence service doesn't act yet. It waits for the Redis TTL to expire (up to 30 seconds). This grace period handles the most common case: brief network blips where the user reconnects within seconds.

If the client reconnects within the TTL window, it immediately sends a heartbeat, refreshing the key. From the outside, the user was never "Away." If they don't reconnect, the key expires and they naturally transition to Away.

Reconnect storms — the thundering herd

When a city-wide internet event ends (stadium concert, power outage, subway emerges from underground), millions of Slack clients attempt to reconnect simultaneously. Without mitigation this creates a thundering herd against the gateway layer.

Slack clients implement exponential backoff with full jitter: reconnect delay = min(cap, base * 2^attempt) with a random component. Each client waits a random 0–60 seconds before its first reconnect attempt, spreading a million-device storm over a minute. Gateways also implement per-IP rate limiting with Retry-After headers.

App backgrounding on mobile

iOS and Android aggressively kill background network connections to save battery. When Slack goes to the background, the WebSocket is typically suspended within 30 seconds. The presence service handles this by having mobile clients explicitly send a BACKGROUND presence event before suspension — but this isn't reliable (the OS may kill the process without warning).

The TTL fallback handles it: if no heartbeat arrives for 30 seconds, the mobile device key expires. If the user is still active on desktop, the aggregated status remains Active. If all devices go silent, they go Away — correctly.

Duplicate heartbeats

On reconnect, a client might send multiple heartbeats in quick succession as it re-establishes state. The presence service handles this idempotently: SET ... EX 30 is naturally idempotent — setting the same key twice just refreshes the TTL. No deduplication logic needed.

How the architecture evolves with scale

Drag the slider to see which components get introduced as user count grows:

1K users100K10M100MEnterprise
Architecture
Single node
Presence store
Single Redis
Heartbeat rate
200 writes/s
Single server handles all WebSocket connections. Redis runs on the same box. Heartbeats at 1K users are trivial — 200 writes/second. Fine for internal tools or early-stage products.

Why not just poll for presence?

Approach Latency Write load Battery impact Handles silence?
HTTP Polling High (1–5s) Extreme Very high No — needs timeout logic
Long Polling Medium (200ms–2s) High Medium Partial
SSE Low Low (server push) Low No — one-way only
WS + Heartbeat ✓ Very low (<100ms) Low Low Yes — TTL expiry

Long polling looks tempting — it's simpler than WebSockets and reasonably low latency. But it fundamentally cannot detect silence: if the client disappears mid-long-poll, the server just has a hanging request with no way to know the user is gone. You'd need a separate timeout mechanism, which is exactly what Redis TTL gives you natively with WebSockets.

SSE (Server-Sent Events) handles the push direction well, but presence requires the client to send heartbeats — SSE is strictly server-to-client. You'd need a hybrid with separate HTTP POSTs for heartbeats, which eliminates most of SSE's simplicity advantage.

How to ace this in a system design interview

What gets stored — and what never touches disk

Presence data is intentionally ephemeral and in-memory. Nothing about "Alice is Active" ever touches a database. Here's the full schema picture — click any table to explore its fields and design rationale.

← Click a table to inspect its schema

Key insight: The presence_state store is Redis — not a relational table. It lives entirely in memory with a TTL. If Redis restarts, all presence data is gone. That's intentional — stale presence (showing someone Active who isn't) is worse than briefly showing everyone Away during a restart.

Redis key structure

-- Per-device presence key (expires if no heartbeat)
SET presence:{userId}:{deviceId} "active" EX 30

-- Aggregated presence (active on any device = active)
SET presence:agg:{userId} "active" EX 35

-- Subscribe to workspace presence changes
SUBSCRIBE presence:ws:{workspaceId}

-- Publish a status change event
PUBLISH presence:ws:{workspaceId} '{"userId":"u_123","status":"away","ts":1700000000}'

-- Heartbeat refresh (idempotent)
SET presence:{userId}:{deviceId} "active" EX 30 XX

-- Explicit away (app backgrounded)
DEL presence:{userId}:{deviceId}

The full system at a glance

The HLD shows every major service boundary, how data flows across them, and which components can fail independently. Use the layer toggle to inspect each tier.

Client layer Edge / Gateway Presence layer Storage layer Async / best-effort

Inside the presence service — sequence by sequence

The LLD walks through the exact method calls, state transitions, and message formats as a heartbeat travels end to end — from app focus to a teammate's green dot update. Step through each phase:

1 / 6

Every design decision has a cost

The choices Slack made aren't universally correct — they're optimised for their specific constraints. Use the radar chart to compare approaches, then explore each decision below.

WS + Heartbeat Long Polling HTTP Polling
Next deep dive →
How Google Calendar Detects Free Slots