How to Build a Real-Time Chat App That Scales

⚡What you are building and why it is harder than it looks

A chat system has four subsystems that look like one feature: a transport layer moving bytes live, a persistence layer making messages durable and queryable, a presence layer tracking who is online, and a delivery layer guaranteeing messages arrive exactly as intended. The demo wires the first one and ignores the rest. Production is the rest.

The difficulty is that chat combines the worst properties of two worlds. Like a database, it must never lose or reorder what was written. Like a streaming system, it must push to thousands of waiting connections in milliseconds. And unlike both, its clients are phones that sleep, switch networks mid-sentence, and come back asking "what did I miss?".

The scaling math is also unforgiving in a specific way: reads and writes decouple violently. One message in a five-hundred-person channel is one write and five hundred deliveries, and a burst in a large channel is a fan-out storm.

🔌Transport: WebSocket vs SSE vs polling, honestly

WebSockets are the right default for chat: one persistent, full-duplex connection per client, low overhead per message, and wide support across browsers and mobile. Their costs are operational rather than technical — load balancers must handle long-lived connections, deployments need connection draining, and a reconnect storm after an outage is a load event of its own. None of these are reasons to avoid WebSockets; they are the work of running them.

Server-Sent Events deserve more respect than they get. SSE is a one-way stream from server to client over plain HTTP, with automatic reconnection built into the browser. For products where the client mostly listens — live activity feeds, notification streams, one-to-many announcements — SSE is simpler to operate, survives proxies and CDNs that mangle WebSocket upgrades, and the client sends its occasional writes as ordinary POSTs. Where SSE loses is true bidirectional chat: sending over POST while receiving over SSE means two channels to keep consistent, and mobile browser support has sharper edges.

Long polling is the fallback, not the foundation: it works through every proxy and firewall on earth, and that is its entire virtue. Keep it as the degradation path for the small share of networks that block upgrades — corporate proxies remain the usual suspects — and treat it as a compatibility mode, not an architecture. One more honest note: managed real-time services (Ably, Pusher, PubNub, Firebase-class backends) genuinely solve the transport and scaling layers below, at per-connection pricing. For a product whose core value is not chat, buying the transport is frequently the correct engineering decision.

🫀Connection management: heartbeats, reconnects, and the resume problem

A connection layer that works at a hundred clients fails at ten thousand in ways that have nothing to do with throughput. Half-open connections — where the network died but neither side noticed — accumulate silently until your connection counts are fiction. The fix is application-level heartbeats: the client pings on an interval, the server drops connections that miss their window, and both sides learn the truth within seconds instead of trusting TCP timeouts measured in minutes.

Reconnects need design, not defaults. Clients reconnect with exponential backoff and random jitter, because a server that restarts with fifty thousand clients will otherwise receive fifty thousand simultaneous reconnections and fall over again — the thundering-herd failure that turns a blip into an outage. Connections should also carry a resume token: when the client comes back, it presents the ID of the last message it saw, and the server replays what it missed from a short-lived buffer.

That resume buffer is the seam between transport and persistence, and sizing it is a product decision: seconds of history covers a network switch, minutes covers a phone sleeping, and anything longer belongs to the "fetch history from the database" path, not the replay buffer. Mobile deserves a final note — on iOS and Android the OS owns background networking, so push notifications carry the wake-up and the socket resyncs on foreground as the normal path, not the error path.

🗄️Message persistence: write first, fan out second

The cardinal rule of chat persistence: a message does not exist until it is durable. The write path is validate, persist, acknowledge to the sender, then fan out to recipients. Systems that fan out before persisting are fast in demos and lose messages in production, because the crash between "delivered" and "written" is real and eventually happens.

Data shape matters more than database brand. Messages are append-only, immutable, and queried overwhelmingly by "give me the N messages in this channel before this cursor" — a workload Postgres handles well with the right indexes, and that wide-column stores like Cassandra or ScyllaDB were practically designed for at extreme scale. Start on Postgres; the access pattern is honest and the operational simplicity is worth more than theoretical headroom you do not need yet. Shard by channel when a single hot channel outgrows one writer — and not before.

Pagination is cursors, never offsets — offsets skip and duplicate as new messages arrive. Every message carries a monotonic sequence within its channel (a database sequence or a snowflake-style ID), which gives you ordering, gap detection, and the resume tokens from the previous section in one mechanism. And decide the retention story early: message history is unbounded growth by definition, so retention windows, export, and deletion are schema decisions, not afterthoughts — especially once enterprise customers ask about retention policies in procurement.

🟢Presence: a distributed-systems problem wearing a green dot

Presence looks like a boolean and behaves like a consensus problem. A user can be connected from three devices, drop one without notice, and the system must answer "is this user online?" correctly under concurrent join and leave events across multiple servers. The standard shape: connection events write to a shared store — Redis is the common choice, with expiring keys refreshed by heartbeats — and presence state derives from "at least one live connection entry exists."

The subtleties are in the edges. Expiring keys mean presence is eventually consistent by design — a crashed client shows online until its key times out, so pick the TTL and heartbeat interval as a product decision (tens of seconds is typical) rather than a bug to eliminate. Fan-out of presence changes should be scoped: users care about their contacts and current channels, not the whole user base, and broadcasting every join and leave to everyone is how presence becomes your highest-volume message type.

Typing indicators and read receipts follow the same pattern at lower stakes: ephemeral, fan-out-scoped, and safe to drop under load. That last property is worth engineering deliberately — mark message classes as droppable so a traffic spike degrades typing indicators instead of chat itself.

📡Horizontal scaling: Redis pub/sub and the adapter pattern

One server holds every connection it accepts, which works until the number of connections exceeds what one process can hold — and the first time you deploy with zero downtime you already need two. The moment a second server exists, so does the core problem of scaled chat: a user connected to server A sends a message to a channel whose members are connected to servers B, C, and D.

The solution is a broadcast backbone: every server publishes outbound messages to a shared bus, and every server delivers to the connections it holds. Redis pub/sub is the workhorse — each server subscribes to the channels its local connections care about, or to a global topic with local filtering, and the socket libraries you are likely already using (the Socket.IO Redis adapter is the canonical example) implement exactly this pattern. Redis Streams or Kafka step in when you need the bus itself to be durable — replayable after a server restart — rather than fire-and-forget.

Two operational rules carry most of the weight at scale. First, shard the bus before it saturates: a single Redis pub/sub instance moves an impressive amount of traffic, but channel-keyed sharding is a straightforward step when the time comes. Second, make connection state portable — sessions, auth, and resume tokens live in shared storage, never in process memory — because the whole point of the architecture is that any server can die and its clients land on a healthy one without noticing anything but a brief resync.

Web development services at Codazz→

📬Delivery guarantees: at-least-once, idempotent, and ordered enough

Exactly-once delivery does not exist over real networks — what exists is at-least-once delivery plus deduplication, and chat is the friendliest possible place to implement it because clients are stateful and messages have natural IDs. The sender generates a client message ID before sending; the server deduplicates on it, so a retried send after a lost acknowledgment cannot create a duplicate. The server acknowledges only after persisting, so a missing acknowledgment always means "retry," never "wonder."

On the receive side, sequence numbers per channel let clients detect gaps and request the missing range, converting "the network dropped a push" from message loss into a brief sync. Duplicates on the receive side are handled the same way as on the send side: clients keep the highest sequence seen and ignore repeats. None of this is exotic — it is the same idempotency discipline as payment processing, applied to a chattier workload.

Ordering deserves honesty: total ordering across a whole workspace is a distributed-transaction problem you do not want; causal ordering within a channel is what users actually perceive. Per-channel sequences give you exactly that, and cross-channel ordering anomalies — a reply appearing to arrive before its question in a different channel — are invisible in practice. Build ordering guarantees per channel and stop there.

🛡️Moderation: build the hooks before you need them

If your chat has strangers in it, moderation is not a phase-two feature — it is the difference between a community and a liability, and retrofitting it into a message pipeline that assumed all content is welcome is genuinely painful. The architectural hooks are cheap if built early: every message passes through a pre-persistence validation step, every message carries a status field (visible, flagged, removed), and removal is a state change that propagates to clients as a normal event, not a deletion that breaks history.

The moderation stack itself is layered. Automated first-pass classifiers — toxicity and spam models, rate-based flood detection, link and media scanning — handle volume. User reporting feeds a review queue for human judgment. And admin tooling handles the account-level actions: mute, suspend, remove. For communities with minors, regulated content, or brand risk, human review is not optional, and the tooling for reviewers — context, history, one-click actions — determines whether moderation keeps up with volume.

Keep one honest principle in view: automated classifiers have false positives, so every automated action needs an appeal path and every removal needs an audit record. Moderation systems that cannot explain their own decisions become the controversy they were built to prevent. Budget the admin tooling into the build, because it is roughly the cost of the chat pipeline itself done properly — and it is the part operators live in every day.

💰Realistic costs and when to hire a team

On cost, as honest labelled ranges. Chat as a feature inside an existing product — channels, persistence, presence, WebSocket transport, basic moderation hooks, built on managed infrastructure — typically adds $40,000 to $90,000 and six to ten weeks to a build. Chat as the product — custom protocol work, resume semantics, at-least-once delivery, presence at scale, full moderation tooling, and load-tested horizontal scaling — typically runs $150,000 to $350,000 over five to eight months. The wide spread reflects the single biggest variable: how much of the transport layer you buy versus build.

Load testing is not optional scope in either tier. Chat systems pass functional tests and fail at concurrency, so budget a real load-testing phase — simulated connections in the tens of thousands, fan-out storms on large channels, reconnect storms after a simulated deploy — before any launch with real users. The failure modes you are testing for do not appear at small scale, which is exactly why teams skip the phase and meet them in production.

Hire a team that has operated chat at scale when concurrency will pass the low thousands, when chat is the product rather than a feature, when moderation carries legal exposure, or when your team has not run long-lived connection infrastructure before. At Codazz, our web development team builds real-time systems on the patterns in this guide — persist-before-acknowledge, per-channel sequences, Redis-backed fan-out — and we will tell you honestly in the first conversation whether your scale justifies building the transport or buying it.

Build your real-time product with Codazz→Load testing guide: k6, JMeter and performance tips→