REALTIME ARCHITECTURE V3 — Multi-Pathway, Actor-Aware, Cost-Bounded

Status: deployed — the waves landed on main (cvl-node-tree-direct merged 2026-05-16). Supersedes the hot-path sections of the retired REALTIME-ARCHITECTURE-V2.md (pruned from the wiki; historical reference only). The 6-layer cockroach scaffolding (L0–L5) survives; this document narrows which layers fire on which path.

Author motivation: the v2 active path makes Cloudflare Durable Objects a hard dependency. At 1M users with ~10 actors per workspace this design does not fit free-tier compute and quietly bleeds cost. V3 reshapes the hot path so DO usage scales with multi-human concurrency, not user count.

Status — Implementation

What landed via cvl-node-tree-direct (merged into main 2026-05-16) and is deployed. The spine (D1 source-of-truth → broker as a thin byte-relay → SDK in @cv/workspace-client) is in place; the hot-path squeeze items shipped in waves.

Deployed

Still proposed

Principles

1. D1 is the source of truth. Every meaningful write lands in D1 via a REST endpoint first. DO outage cannot lose data.
2. Agents do not hold WebSockets. Agents are the majority of actors in a typical workspace. They speak REST/MCP and read D1 on tool-call boundaries. Optionally subscribe to a per-workspace push stream when they want low-latency awareness.
3. DO fans out logical changes, not Yjs micro-ops. "Drag job to status X" is one broadcast, not N keystroke events. Yjs micro-op rate stays peer-local (P2P or one process).
4. Yjs over WS only when ≥2 humans are co-editing a Yjs-relevant surface (text editors: notes, CVL). Otherwise REST + change-feed is enough.
5. Multi-pathway resilience. Every mutation has at least two delivery channels: durable (D1 via REST) and live (DO / P2P / SSE / relay list). Either alone keeps the user safe.
6. Vendor-portable wire format. Plain WS frames carrying Yjs sync. No Cloudflare-specific framing. Failover to a non-CF relay is a config change, not a rewrite.

Workload assumptions

Anchored in project_careervector_workspace_actor_model.md and project_careervector_scale_target.md:

• ~10 actors per workspace typical: 1–3 humans + ~7–9 agents (Codex, Claude, Gemini, …).
• ~100k workspaces at 1M users.
• ~5 active min/day per human × 22 days/month.
• Multi-human concurrent moments ≈ 5% of active time. The other 95%: one human + N agents.
• Agents poll/subscribe at tool-call rhythm, not human-keystroke rhythm.

Architecture (hot path)

                ┌──────────────────────────────────────────┐
                │       D1 (canonical state)               │
                │   ← writes ← REST ← every actor          │
                │   change_log table tracks logical edits  │
                └─┬────────────────────────────┬───────────┘
                  │                            │
       change_log change-feed         read on demand
                  │                            │
        ┌─────────▼──────────┐      ┌──────────▼─────────┐
        │ SSE Fanout Worker  │      │ Agents (REST/MCP)  │
        │ stateless, per-ws  │      │ poll on tool calls │
        │ stream of {clock,  │      │ never open WS      │
        │ kind, path}        │      └────────────────────┘
        └─────────┬──────────┘
                  │
                  │ (agents that opt into push)
                  │
                  ▼
        ┌────────────────────────────────────────────────┐
        │                Humans                          │
        │  ┌─────────────────────────────────────┐       │
        │  │ Browser leader tab                  │       │
        │  │ Y.Doc + IndexedDB                   │       │
        │  │ BroadcastChannel ↔ follower tabs    │       │
        │  └──┬─────────────────┬────────────────┘       │
        │     │                 │                        │
        │     ▼ P2P (humans)    ▼ WS (signaling +        │
        │  ┌──────────┐         │      logical changes)  │
        │  │ y-webrtc │    ┌────▼──────────────────┐     │
        │  │ Yjs ops  │    │ RelayWalker picks     │     │
        │  │ peer↔peer│    │ the current entry     │     │
        │  └──────────┘    │ from PUBLIC_RELAY_URLS│     │
        │                  │ and advances on       │     │
        │                  │ failure               │     │
        │                  └───────┬───────────────┘     │
        │                          │                     │
        │                          ▼                     │
        │   ┌─────────────────────────────────────────┐  │
        │   │ cf  : CF DO (`realtime`, cf.relay.…)    │  │
        │   │       - signaling                       │  │
        │   │       - logical-change broadcast        │  │
        │   │       - hibernates idle                 │  │
        │   │ deno: Deno Deploy (`relay/deno/`,       │  │
        │   │       deno.relay.…) — same protocol     │  │
        │   │ …   : append more vendors with their    │  │
        │   │       own <vendor>.relay.… prefix       │  │
        │   └─────────────────────────────────────────┘  │
        └────────────────────────────────────────────────┘

The four load-shedding rules

Rule 1 — Agents do not open WebSockets

The dominant population of actors. Currently each agent connecting via the GUI client gets its own WS. New rule:

• Agent runtimes hit /api/... REST + /mcp/... for tool calls.
• Agents that want push notifications open one shared SSE stream per workspace at /api/workspaces/:id/changes/stream. The stream emits {clock, kind, path} for every logical change. Agents fetch full state via REST when they care.
• One stream per agent process per workspace, not per tool-call. Agents reconnect on disconnection.

Effect: ~80% of actor-connections leave the WS layer.

Rule 2 — Browser opens WS only when needed

The leader tab evaluates on focus / visibility-change:

shouldOpenWs = (peerHumansPresent >= 2) || (currentSurface in {notes, cvl-editor})

peerHumansPresent is read from a lightweight /api/workspaces/:id/presence endpoint (D1 read, ~10ms). Polled lazily, not held open.

Otherwise the tab uses REST + the same SSE stream to receive logical-change notifications and re-fetch via REST. Solo-human-with-agents sessions never open a WS.

Effect: most active sessions cost zero DO compute.

Rule 3 — DO broadcasts logical changes, not Yjs micro-ops

Two distinct broadcast types, only one of which goes through DO:

Coalescing happens client-side with a 250–500 ms ring buffer per logical surface. Aggregates rapid edits into a single "logical change" event.

Effect: ~10–50× fewer DO messages during active editing.

Rule 4 — REST is the durable backstop

Every mutation:

client.mutate(value):
  PUT /api/workspaces/:id/...    ← D1 write, returns clock N
  if (ws-connected):
    broadcast({clock: N, kind, path})  ← live notification, optional
  insert into change_log (clock, kind, path)  ← server-side, in REST handler

DO outage = no live sync; data still safe in D1. Reload recovers from D1. Agents catch up on next poll.

Resilience: multi-vendor relay (list-walker)

The realtime relay protocol is plain WS frames + the Yjs sync subset that v2 already speaks. No CF-specific extensions on the wire.

The browser ships with an ordered list of relay endpoints and walks it on failure. There is no special "primary vs alt" binary — every entry is a peer relay with the same protocol, and the list defines the failover order.

Configuration (single env var)

PUBLIC_RELAY_URLS is a build-time JSON array of {label, url} objects, ordered by priority. Each url is a WebSocket URL ending in /ws (the server mounts /ws/<workspaceId>); each label is a stable, unique key that doubles as the debug-pin key and the cockpit-probe rollup key.

Production value (ui/wrangler.toml [env.production.vars], mirrored in ui/vite.config.ts):

[
  { "label": "cf",   "url": "wss://cf.relay.careervector.corbet.ch/ws" },
  { "label": "deno", "url": "wss://deno.relay.careervector.corbet.ch/ws" }
]

Vendor-prefix convention

Relay hostnames live under <vendor>.relay.careervector.corbet.ch. The vendor prefix matches the relay's label so an operator who reads a telemetry line for label=cf can find the matching hostname without a translation table. Adding a third relay = pick a new vendor key (e.g. fly), provision fly.relay.careervector.corbet.ch, append to PUBLIC_RELAY_URLS.

Client walker (RelayWalker)

Each WorkspaceConnection owns a per-tab RelayWalker (ui/src/lib/realtime/relay-walker.ts). The walker:

1. Starts at index 0.
2. Advances when EITHER:
   • The WebSocket fails before onopen fires (close before-or-during handshake, pre-open error), OR
   • Two consecutive abnormal closes follow a successful open. Clean closes (code 1000) reset the post-open strike counter.
3. After the last entry, wraps to 0 AND emits a relay_cycle_all_failed telemetry event so operators learn every relay is failing.
4. Persists position across reconnect cycles within the tab (does not reset on clean close).
5. Emits a relay_advance telemetry event per transition tagged with fromLabel, toLabel, and trigger.

Debug override (?relay=<label>)

Appending ?relay=cf (or any other known label) to the SK page URL pins the walker to that entry for the lifetime of the tab. While pinned:

• The walker stays on the pinned entry through any number of failures (broken stays broken — the operator sees the real issue).
• The cycle telemetry never fires.
• Unknown labels: console.warn + fall back to default selection.

The pin is per-tab; opening a new tab without the param gets the normal walker behaviour. Useful for diagnosing whether a specific relay is the cause of a connectivity problem.

Math at 1M users (back of envelope)

• 100k workspaces × 5% multi-human-concurrent active time × 110 active min/month × 2 logical-change broadcasts/min ≈ 1.1M DO broadcasts/month.
• Free-tier DO ceiling: 1M req/month. Sits at ceiling without paid plan.
• Workers Paid ($5 base + $0.15/M extra) at 10× this volume: ~$5–10/mo.
• Cost grows linearly with active multi-human concurrency, not user count. A 10× user growth that keeps the same concurrency profile costs ~10× nothing.

D1 change_log writes: ~2 broadcasts/active-user-min × 100k users × 5 min/day ≈ 1M writes/day. Above free-tier D1 write budget (50k/day). Mitigations:

• TTL the change_log to 7 days.
• Co-locate the change_log row write inside the existing mutation transaction (no extra row in many cases — encode as a column on the main table).
• Compress events: (workspace_id, clock, kind, path) keyed only by clock; many event kinds collapse to a counter.

Implementation plan

Total ~3.5 days of focused work. No rewrite of the cockroach scaffolding. Both PHASE_4_THIN_RELAY=1 and the existing legacy mode survive untouched; V3 is additive on the client side.

Approved squeeze list (2026-04-28)

These compound. Each multiplies headroom on top of the baseline cost. Implementation state marked per row.

Combined: cost baseline drops to ~$2–3/mo at 1M users, free-tier headroom ~10× more active concurrency.

#1, #2, #3, #4, #10 hard-bake into the SDK from day one. #5, #6, #7, #8, #9 land incrementally.

Reconciled cost model at 1M users / 100k DAU

$0.000008/user/month. With squeeze items #1–4 applied, drops to ~$2–3/month.

Cost shape scales linearly with active multi-human concurrency, not user count.

Three disciplines enforced by SDK + linter

1. No background polling cadence by default. Polling is event-driven: focus, action, idle-tick at minutes (not seconds). The SDK's subscribe callback only triggers re-fetch on these events, not on a timer. Lint rule: no setInterval in any code path that touches the workspace API.
2. All mutations through applyOp. No direct fetch('/api/workspaces/...'), no direct Y.Doc construction outside the SDK. Lint rule: ban these patterns in app code.
3. Coalesce to logical units before writing. One D1 write per "user finished doing something." The Yjs ops still flow per-keystroke for P2P sync; D1 only sees coalesced units. Lint rule: every applyOp call site must pass through the SDK's coalescing buffer.

If any discipline breaks, the cost shape breaks. The linter is load-bearing.

Implementation plan (PR-shaped)

Open questions

• Where exactly does the SSE Fanout Worker live? Cloudflare Workers cap long-lived streams at ~30s on free tier; on paid, streams are billable for the duration. Hetzner / Koyeb may be cheaper per stream-hour.
• D1 change_log schema: separate table vs inline column? Decide based on benchmark of write amplification.
• y-webrtc signaling channel: still goes through DO. Is a non-DO signaling path (HTTP-based ICE candidate exchange) feasible? Already partially in v2 (/api/p2p/signal) — promote to primary.
• TURN bandwidth cost (Cloudflare Calls T2 tier) when humans are behind tight NATs and P2P falls back through TURN.
• Multi-vendor failover semantics: what state does a fallback relay need to bootstrap from? Probably nothing — same Yjs sync from each peer's local Y.Doc.

Relationship to V2

V2's 6-layer cockroach scaffolding survives:

REALTIME-ARCHITECTURE-V2.md (retired — pruned from the wiki in 2cbd1969) was the design doc for the byte-relay mechanics. V3 is the workload-shaping layer on top of it.