Server-Sent Events (SSE) vs WebSockets: Streaming Architecture for LLM & Real-Time APIs

The rise of Large Language Models (LLMs) and generative AI interfaces has fundamentally shifted real-time web architecture. While full-duplex messaging protocols like WebSockets dominated the chat and gaming era, unidirectional streaming via Server-Sent Events (SSE) has emerged as the de facto standard for token-by-token text generation (OpenAI, Anthropic, Gemini, DeepSeek), live metrics streaming, and server notification pipelines.

Choosing between SSE and WebSockets requires understanding transport layer trade-offs, proxy and CDN buffering behaviors, firewall traversals, and connection concurrency limits under HTTP/1.1 vs HTTP/2 and HTTP/3.

This guide provides a comprehensive comparison of SSE and WebSockets, dissects the text/event-stream wire format, and provides production-ready server and client implementations across Node.js/Next.js, Python (FastAPI), and Go.

---

1. Architectural Comparison: SSE vs WebSockets

┌─────────────────────────────────────────────────────────────────────────────┐
│                          Protocol Architecture                              │
├──────────────────────────────────────┬──────────────────────────────────────┤
│      Server-Sent Events (SSE)        │              WebSockets              │
├──────────────────────────────────────┼──────────────────────────────────────┤
│ • Transport: Standard HTTP (1.1 / 2) │ • Transport: TCP upgraded from HTTP  │
│ • Direction: Unidirectional (S → C)  │ • Direction: Full-Duplex (C ⇄ S)     │
│ • Data Framing: UTF-8 Text (chunks)  │ • Data Framing: Text or Binary frames│
│ • Reconnection: Built-in native auto │ • Reconnection: Manual client logic  │
│ • Multiplexing: Native over HTTP/2   │ • Multiplexing: Separate TCP streams │
│ • Firewall/Proxy: Traverses natively │ • Firewall/Proxy: May require config │
│ • Primary Use: LLM tokens, Feeds     │ • Primary Use: Gaming, Collab-canvas │
└──────────────────────────────────────┴──────────────────────────────────────┘

Key Differences Breakdown

1. Directionality:

   • SSE: Unidirectional from server to client. The client initiates a standard HTTP GET request, and the server keeps the connection open, pushing text chunks as they become available.
   • WebSockets: Bidirectional. After a TCP upgrade handshake (101 Switching Protocols), both client and server can send messages simultaneously across the socket without HTTP request overhead.

2. HTTP/2 & HTTP/3 Multiplexing:

   • Under HTTP/1.1, browsers enforce a strict limit of 6 concurrent open connections per origin. Holding an SSE connection open consumes 1 of those 6 slots, causing subsequent API calls to queue.
   • Over HTTP/2 or HTTP/3, SSE streams are multiplexed over a single TCP/QUIC connection, eliminating the 6-connection barrier.
   • WebSockets bypass HTTP/2 multiplexing entirely: every active WebSocket connection maintains an independent TCP connection.

3. Wire Protocol & Framing:

   • SSE: Plain UTF-8 text framed by newlines (data: ...\n\n), easily inspected with browser DevTools, cURL, or proxy loggers.
   • WebSockets: Binary framing protocol with masking keys, opcodes (0x1 for text, 0x2 for binary, 0x8 for close, 0x9 for ping), requiring specialized frame decoders.

---

2. The SSE Wire Format (text/event-stream)

An SSE stream is served with the Content-Type: text/event-stream header, along with Cache-Control: no-cache and Connection: keep-alive.

Each message consists of one or more fields formatted as field: value\n, terminated by a double newline (\n\n):

HTTP/1.1 200 OK
Content-Type: text/event-stream; charset=utf-8
Cache-Control: no-cache, no-transform
Connection: keep-alive
X-Accel-Buffering: no

event: token
id: msg-101
retry: 3000
data: {"content": "Hello", "finish_reason": null}

event: token
id: msg-102
data: {"content": " world!", "finish_reason": null}

event: done
id: msg-103
data: [DONE]

Protocol Fields Defined by the HTML Living Standard

• data: The actual payload. Multi-line payloads must prefix every line with data: (e.g. data: line1\ndata: line2\n\n).
• id: Event identifier. If the connection drops, the browser automatically sends this ID in the Last-Event-ID request header upon reconnecting, allowing seamless state resumption.
• event: Custom event name. Clients listen to specific types using eventSource.addEventListener('eventName', handler). Defaults to message.
• retry: Reconnection timeout in milliseconds. Instructs the client how long to wait before attempting to reconnect if the stream disconnects unexpectedly.
• : comment: Lines starting with a colon are treated as comments or heartbeat pings to prevent network timeouts without triggering client message handlers.

---

3. Implementing LLM Streaming with SSE

Server: Next.js App Router (app/api/chat/route.ts)

import { NextRequest } from 'next/server';

export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';

export async function POST(req: NextRequest) {
  const { prompt } = await req.json();

  // Create a TransformStream for SSE encoding
  const encoder = new TextEncoder();
  const stream = new TransformStream();
  const writer = stream.writable.getWriter();

  (async () => {
    try {
      const mockTokens = ['Modern ', 'web ', 'architectures ', 'leverage ', 'SSE ', 'for ', 'streaming.'];
      let messageId = 1;

      for (const token of mockTokens) {
        // Construct SSE payload
        const payload = JSON.stringify({ token, timestamp: Date.now() });
        const sseChunk = `id: ${messageId++}\nevent: token\ndata: ${payload}\n\n`;
        
        await writer.write(encoder.encode(sseChunk));
        await new Promise((res) => setTimeout(res, 80)); // Simulate LLM generation latency
      }

      // Send completion event
      await writer.write(encoder.encode('event: done\ndata: [DONE]\n\n'));
    } catch (err) {
      const errorPayload = JSON.stringify({ error: 'Stream failed' });
      await writer.write(encoder.encode(`event: error\ndata: ${errorPayload}\n\n`));
    } finally {
      await writer.close();
    }
  })();

  return new Response(stream.readable, {
    headers: {
      'Content-Type': 'text/event-stream; charset=utf-8',
      'Cache-Control': 'no-cache, no-transform',
      'Connection': 'keep-alive',
      'X-Accel-Buffering': 'no', // Critical for Nginx
    },
  });
}

Client: Modern fetch Streaming Reader (Consuming POST SSE)

The native browser EventSource API only supports GET requests without custom authorization headers. For POST-based LLM streaming with Bearer tokens, use the Web Streams API with fetch:

async function streamChatCompletion(prompt: string, token: string, onToken: (t: string) => void) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${token}`,
      'Accept': 'text/event-stream',
    },
    body: JSON.stringify({ prompt }),
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP Error: ${response.status} ${response.statusText}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder('utf-8');
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split('\n');
    buffer = lines.pop() || ''; // Keep partial line in buffer

    for (const line of lines) {
      const trimmed = line.trim();
      if (trimmed.startsWith('data: ')) {
        const dataStr = trimmed.slice(6);
        if (dataStr === '[DONE]') return;
        
        try {
          const parsed = JSON.parse(dataStr);
          if (parsed.token) onToken(parsed.token);
        } catch {
          // Non-JSON or raw text data chunk
          onToken(dataStr);
        }
      }
    }
  }
}

---

4. Reverse Proxy & Infrastructure Gotchas

Nginx Proxy Buffering

By default, Nginx buffers upstream responses until the buffer fills (e.g. 4KB or 8KB). This completely breaks real-time streaming, resulting in the entire LLM response arriving in a single batch after completion.

Fix via Nginx Config:

location /api/streaming/ {
    proxy_pass http://upstream_backend;
    
    # Disable buffering for real-time SSE streaming
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header Connection '';
    proxy_http_version 1.1;
    chunked_transfer_encoding on;

    # Extend timeouts for long-running generative models
    proxy_read_timeout 600s;
    proxy_send_timeout 600s;
}

Fix via Application Header: Return X-Accel-Buffering: no in your HTTP response headers. Nginx recognizes this response header and automatically disables buffering for that specific stream.

Cloudflare & CDN Compression

Some CDNs attempt to gzip or compress streaming responses on the fly, holding chunks in memory until a compression block is satisfied. Ensure your CDN rules disable caching and transformation for routes returning Content-Type: text/event-stream or pass Cache-Control: no-transform.

---

5. Decision Matrix: When to Choose Which

┌───────────────────────────────────────┬──────────────┬─────────────┐
│ Requirement / Use Case                │ Preferred    │ Reason      │
├───────────────────────────────────────┼──────────────┼─────────────┤
│ LLM Token Streaming / AI Completion   │ SSE          │ Unidirectional, HTTP/2 multiplexed, easy to debug │
│ Live Stock / Crypto Tickers           │ SSE          │ Server push only, low protocol overhead            │
│ Server Notifications & In-App Alerts  │ SSE          │ Built-in auto-reconnection via Last-Event-ID      │
│ Real-Time Multiplayer Gaming          │ WebSockets   │ Sub-millisecond bidirectional binary messaging    │
│ Collaborative Canvas (Figma-style)    │ WebSockets   │ Simultaneous cursor/state broadcast across peers  │
│ Peer-to-Peer Signaling (WebRTC)       │ WebSockets   │ Low-latency full-duplex session negotiation       │
└───────────────────────────────────────┴──────────────┴─────────────┘

---

6. Testing & Debugging Streaming Endpoints

Test streaming endpoints using cURL without buffering:

# Test an SSE stream via cURL (disabling output buffering with -N)
curl -N -X POST https://api.example.com/api/chat \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{"prompt": "Explain SSE in two sentences"}'

You can also simulate and inspect request streams using the API Request Builder and verify webhook triggers with Webhook Tester.