⚠️ HISTORICAL DESIGN. The body below describes the pre-byte-relay broker design (op validation at ingest, DO SQLite persistence, D1 cold backup). It was superseded by the live thin byte-relay: the
WorkspaceDocDO (relay/cf/src/workspace-doc.ts) does NOT parse or validate ops — it fans out raw Yjs sync frames verbatim, and D1 (workspace_sub_doc) is the durable source of truth written by the ui worker. For the current contract, read the frontmatter summary above, REALTIME-ARCHITECTURE-V3.md, and CLAUDE.md §3. The body is retained as design history; do not implement from it.
Realtime Broker Design (Yjs over Durable Object)
The implementation spec for relay/cf/src/workspace-doc.ts. One Durable Object per workspace. Holds the canonical Y.Doc, validates ops at ingest, broadcasts Yjs deltas, persists to DO SQLite, cold-backs-up to D1.
This document is the contract. Do not deviate without updating this file first.
References: ARCHITECTURE.md (Y.Doc structure, persistence layout), PRINCIPLES.md #1 (compute economics), #3 (no fallbacks), #8 (client compute default), #12 (one source of truth). See also REALTIME-ARCHITECTURE-V3.md for the multi-pathway resilience model that wraps this broker.
0. Scope
In scope:
- Lifecycle of one
WorkspaceDocDO instance. - Ingest, validate, apply, persist, broadcast for Yjs document updates.
- Awareness routing (presence, cursors). Awareness is NOT persisted.
- D1 cold backup cadence.
- Failure handling for the cases listed in §7.
Out of scope (handled elsewhere):
- Workspace creation in D1 (web Worker). DO assumes the workspace row exists; the Worker validates before routing (lift v1's zombie gate).
- LLM calls, Typst rendering, exports — all client-side per principle #8.
- Auth — link = identity; no auth surface in the broker.
- MCP server — separate Worker; it talks to this DO via service binding for ops it needs to inject server-side.
1. DO lifecycle
1.1 Instantiation
A WorkspaceDoc instance is materialized the first time any code calls env.WORKSPACE_DOC.get(idFromName(wsId)) and then invokes a method on the stub. The Worker entry already validates the workspace exists in D1 before this call (zombie gate, lifted from v1).
On constructor(state, env) the DO does the minimum:
- Stash
stateandenv. - Initialize the in-memory
Y.Doc. - Restore session metadata for any hibernated WebSockets via
state.getWebSockets()+deserializeAttachment(). - Defer SQLite reads (snapshot + ops log) until the first
fetchorwebSocketMessage. The constructor must stay cheap because it runs on every wake.
1.2 First-touch hydration (hydrate())
Called lazily (idempotent, guarded by a hydrated: boolean flag). Reads the latest snapshot from SQLite, applies it to the Y.Doc, then replays the ops log on top. Pseudocode:
private async hydrate(): Promise<void> {
if (this.hydrated) return;
const snap = this.sql.exec<{ data: ArrayBuffer; clock: number }>(
'SELECT data, clock FROM snapshot ORDER BY clock DESC LIMIT 1'
).one();
if (snap) Y.applyUpdate(this.doc, new Uint8Array(snap.data), 'hydrate');
const cursor = this.sql.exec<{ data: ArrayBuffer }>(
'SELECT data FROM ops WHERE clock > ? ORDER BY clock ASC',
snap?.clock ?? 0,
);
for (const row of cursor) {
Y.applyUpdate(this.doc, new Uint8Array(row.data), 'hydrate');
}
this.hydrated = true;
}
Errors during hydrate are fatal — return 503 to the client. Per principle #3, no fallbacks; corruption surfaces, doesn't get papered over.
1.3 Hibernation
The runtime hibernates the DO when there are no active timers, no pending I/O, and all open WebSockets were acceptWebSocket()-ed (hibernatable mode). What survives:
- DO SQLite (storage class
new_sqlite_classesperwrangler.toml): snapshot + ops log + meta rows persist across hibernation and crashes. - Hibernated WebSockets: connections stay open at the runtime level; sockets are re-bound to a fresh JS object on wake.
- Per-socket attachments (
serializeAttachment()): session metadata (userId, joinedAt) survives.
What does NOT survive:
- The in-memory
Y.Doc. It must be reconstructed on wake from snapshot + ops log (§1.2). - The
Map<WebSocket, Session>view. Rebuild fromstate.getWebSockets() + deserializeAttachment()in the constructor. - Awareness state. Treated as ephemeral by Yjs; clients re-broadcast their awareness on reconnect (and on wake clients discover each other again via fresh sync step 1).
1.4 Wake path
- Constructor runs, restores socket metadata, sets
hydrated = false. - First inbound message (or fetch) calls
hydrate(). Latency budget: see §10. - While hydrating, inbound messages are queued (one per socket, bounded — see §10).
- After hydrate, drain the queue in arrival order. From now on, normal path.
2. Y.Doc canonical state
2.1 Where it lives
The Y.Doc lives in DO memory (this.doc: Y.Doc) while the DO is awake. There is exactly one Y.Doc per workspace per DO instance. Per principle #12, this is the source of truth for the workspace's editable state during the live window. D1 is cold backup, not a parallel writer.
The shape inside the Y.Doc matches ARCHITECTURE.md §"Y.Doc structure per workspace": settings, layout, order_state, cv_profile, cl_profile, jobs, cloud_keys. The broker does not care about substructure beyond schema validation (§8); it stores opaque updates.
2.2 Persistence model
Two SQLite tables in the DO's storage:
-- Append-only log of validated updates since the last snapshot.
CREATE TABLE IF NOT EXISTS ops (
clock INTEGER PRIMARY KEY, -- monotonic op counter, NOT a Yjs clock
data BLOB NOT NULL, -- raw Yjs update bytes
origin TEXT, -- userId from the WS session, or 'mcp', 'system'
ts INTEGER NOT NULL -- ms epoch when applied
);
-- One snapshot row at a time after compaction; older rows kept until R2 archive (out of scope here).
CREATE TABLE IF NOT EXISTS snapshot (
clock INTEGER PRIMARY KEY, -- ops.clock at the moment of snapshot
data BLOB NOT NULL, -- Y.encodeStateAsUpdate(doc)
ts INTEGER NOT NULL,
byte_size INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
-- Keys: 'op_counter', 'last_d1_backup_clock', 'last_d1_backup_ts'.
clock is a DO-local monotonic counter, NOT a Yjs vector clock. It exists so the broker can replay ops in the order they were applied without trusting client-supplied ordering. Yjs handles real causal ordering inside the update bytes.
2.3 Concrete numbers (N, M)
Snapshot when EITHER condition fires:
- N = 200 ops since the last snapshot, OR
- M = 5 minutes of wall-clock time since the last snapshot if any ops were appended in that window.
Reasoning:
- Cold-start cost on wake is dominated by replaying the ops log. 200 small CRDT updates apply in <50ms; that's the pain-free ceiling.
- A typical edit session bursts ~30-80 ops in a minute (typing in fields, reordering sections). N=200 means a snapshot every few minutes during active editing, not every keystroke.
- M=5min catches the long-tail case where a workspace gets one op every 90 seconds — without M, the log would grow unbounded. The 5-minute upper bound on staleness keeps wake latency predictable.
- These numbers are tunable. See §11 open questions.
A snapshot pass is also forced before D1 backup (§6) so the cold backup is always consistent.
3. Connection lifecycle
The broker speaks the standard Yjs sync protocol over a single WebSocket. Reference: y-protocols (specifically y-protocols/sync and y-protocols/awareness), encoded with lib0. Messages are framed as binary Uint8Arrays; the first varint is the message type.
3.1 Message types (Yjs convention)
| varint | y-protocols name | Direction | Purpose |
|---|---|---|---|
| 0 | messageSync |
both | Sync sub-protocol envelope (steps 1, 2, update) |
| 1 | messageAwareness |
both | Awareness state |
| 2 | messageAuth |
server→client | Auth-deny (we never send; link=identity) |
| 3 | messageQueryAwareness |
client→server | Request awareness of all peers |
Inside messageSync, the second varint distinguishes sub-types: 0=SyncStep1, 1=SyncStep2, 2=Update.
3.2 Connect flow
- Client picks a relay entry from
PUBLIC_RELAY_URLSvia itsRelayWalker(ui/src/lib/realtime/relay-walker.ts) and connects to<wsUrl>/<wsId>?userId=<deviceId>(todaywss://cf.relay.careervector.corbet.ch/ws/<wsId>?userId=<deviceId>for the CF entry,wss://deno.relay.careervector.corbet.ch/ws/<wsId>?userId=<deviceId>for the Deno entry). See REALTIME-ARCHITECTURE-V3.md §"Resilience: multi-vendor relay" for the walker contract. - Worker validates
wsIdformat and existence in D1 (lifted from v1 — keep the zombie gate verbatim). - Worker forwards the upgrade to the DO stub. DO calls
state.acceptWebSocket(server)and storesSession { userId, connectedAt }viaserializeAttachment. - DO triggers
hydrate()if not already done. - Server initiates sync: DO sends
SyncStep1(stateVector(this.doc))to the new client. (Yjs allows either side to start; server-initiated keeps the catch-up payload deterministic.) - Client responds with
SyncStep2(diff)containing all updates the server is missing, plus its ownSyncStep1requesting the server's state. - DO answers with its
SyncStep2. New client now has full state; server has client's offline-accumulated ops. - DO sends current awareness snapshot (
messageAwarenesswith all known peer states). - Client sends its initial awareness; DO routes per §4.
3.3 Live phase
While the connection is open:
- Any
messageSync(Update)from the client is validated (§8), applied to the canonical Y.Doc, persisted (§2), and broadcast (§4). - Any
messageAwarenessis applied to the awareness instance and broadcast (§4). - Any
messageQueryAwarenessis answered with the current awareness snapshot back to the asking socket.
3.4 Catch-up after disconnect
Yjs handles this natively via vector clocks — there is no separate "missed ops" channel. On reconnect, the client repeats §3.2 from step 1, the server's SyncStep1 carries its current vector clock, the client's SyncStep2 sends only the deltas the server is missing, and vice versa. No full snapshot transfer unless §7.4 triggers.
4. Broadcast topology
One DO, N WebSockets. Star topology — every applied update fans out to the other N-1 sockets.
4.1 Document updates
When a messageSync(Update) from socket A is validated and applied:
// Pseudocode
const update = readSyncUpdate(decoder);
validate(update); // §8
Y.applyUpdate(this.doc, update, sessionA); // origin tag = sender socket
appendToOpsLog(update, sessionA.userId); // §2.2
maybeCompact(); // §5
broadcastUpdate(update, exceptSocket=A); // fan-out
The fan-out re-encodes the update inside a messageSync(Update) envelope and sends the binary frame to every other open socket. Sender is excluded — it already has the change locally (CRDT property: applying your own update is a no-op).
4.2 Awareness
Awareness updates use messageAwareness framing and Awareness (y-protocols/awareness) state-merge semantics. Awareness is NEVER applied to the Y.Doc and NEVER persisted. It's strictly in-DO-memory + broadcast.
Behavior:
- Apply incoming awareness update to
this.awareness(so the DO can answermessageQueryAwarenessfrom late joiners). - Broadcast verbatim to all other sockets.
- On
webSocketClose, remove the disconnecting client's awareness state and broadcast a removal update.
This is the data path that powers v1's upper-right presence indicator. The UI is feature-complete (principle #13); we're only swapping the wire format.
4.3 Why server-applies (vs pure relay)
v1 was a pure relay: binary frames forwarded blind. v2 applies updates server-side because:
- We need a canonical Y.Doc to persist (snapshot/ops log).
- We need to validate against Zod schemas (§8) before broadcast — invalid ops never reach other clients.
- Late joiners need the server to answer SyncStep1 from a real state, not from "whatever the last client to talk had".
Cost: applying a Yjs update is microseconds. The DO stays well under the 10ms-per-request principle.
5. Snapshot compaction
5.1 Trigger
maybeCompact() runs after every appended op. It checks:
- Op count since last snapshot ≥ N (200), OR
- Wall-clock since last snapshot ≥ M (5 minutes) AND at least one op since.
A setAlarm() covers the M case when the DO would otherwise hibernate before the timer fires; the alarm handler runs maybeCompact() and re-arms if needed.
5.2 Procedure
private async compact(): Promise<void> {
const fullState = Y.encodeStateAsUpdate(this.doc);
const clock = this.opCounter;
const ts = Date.now();
this.sql.transactionSync(() => {
this.sql.exec(
'INSERT INTO snapshot (clock, data, ts, byte_size) VALUES (?, ?, ?, ?)',
clock, fullState, ts, fullState.byteLength,
);
this.sql.exec('DELETE FROM ops WHERE clock <= ?', clock);
this.sql.exec(
'INSERT OR REPLACE INTO meta (key, value) VALUES (?, ?)',
'last_snapshot_clock', String(clock),
);
});
}
transactionSync makes the snapshot insert + ops delete atomic. If the DO crashes between the insert and delete, the next hydrate sees a snapshot at clock K and ops at clock ≤ K; replaying those ops is a no-op for the Y.Doc (CRDT idempotence) but wastes time. We tolerate this — see §7.1 (soft-fail).
5.3 Older snapshots
Older snapshot rows are kept in DO SQLite for 24h then archived to R2 (per ARCHITECTURE.md's "Older snapshots… R2"). The R2 archival cron is OUT OF SCOPE for this design — open question §11.5.
For now: compact() keeps only the most recent snapshot row (the INSERT could be INSERT OR REPLACE keyed on clock if we collapse to single row; we keep the multi-row table to leave room for the archival job to read, then delete). Implementer choice — start with single-row, expand only when archival lands.
6. D1 cold backup
D1 is the durable source of truth per ARCHITECTURE.md and REALTIME-ARCHITECTURE-V3.md. Per principle #1, we don't burn writes that aren't needed.
6.1 Cadence
Backup to D1 ONCE PER HOUR while the DO is awake AND has unsaved changes. Implementation: a DO alarm (state.storage.setAlarm) set on first op after the previous backup, scheduled for lastBackupTs + 1h (or immediately if more than 1h has elapsed). The handler:
- Forces a
compact()so the snapshot row reflects the latest state. - Reads the current snapshot bytes.
- Writes to D1:
UPDATE workspaces SET y_snapshot = ?, y_snapshot_clock = ?, y_snapshot_ts = ? WHERE id = ?. - Records
last_d1_backup_clock+last_d1_backup_tsinmeta. - Re-arms the alarm only if new ops have arrived since the backup completed.
The DO may also hibernate between backups; that's fine because DO SQLite IS durable. D1 is purely the "DO got nuked / migration disaster / human inspection" backup.
6.2 Justification (principle #1)
A typical workspace edited daily: ~24 D1 writes/day for backup, vs the alternative options:
| Cadence | Writes/day | Why rejected |
|---|---|---|
| Every snapshot (N=200 ops) | 50-200/day during active editing | Burns D1 writes for redundancy already provided by DO SQLite |
| On hibernate only | ~5/day | Loses recency in the rare DO-data-loss event; hibernate isn't always graceful |
| Hourly while-active | ~24/day | Sweet spot: recency + cheap |
| Every minute | ~1440/day | Wasteful — DO SQLite IS the live durable store |
D1 free tier is 100k writes/day across the whole account. Hourly per workspace scales to 4000 active workspaces before we'd need to revisit, which is far beyond v1's reality.
6.3 D1 schema delta
Add to existing workspaces table (cold-backup columns; nullable to coexist with v1):
ALTER TABLE workspaces ADD COLUMN y_snapshot BLOB;
ALTER TABLE workspaces ADD COLUMN y_snapshot_clock INTEGER;
ALTER TABLE workspaces ADD COLUMN y_snapshot_ts INTEGER;
Migration tracked separately. The web Worker reads y_snapshot to bootstrap a fresh client load before WebSocket sync (per ARCHITECTURE.md §"Read-only views").
7. Failure modes
7.1 DO crash mid-write (soft-fail)
Yjs updates are CRDT — applying the same update twice is idempotent, applying overlapping updates converges. So:
- If the DO crashes between
Y.applyUpdateandappendToOpsLog: the in-memory Y.Doc is gone anyway; on next hydrate the update is lost UNLESS the client retries. Clients DO retry: Yjs's IndexedDB persistence holds the update locally and the next sync round re-sends it. - If the DO crashes between
appendToOpsLogandbroadcast: persisted but other clients didn't see it live. They learn it on their next sync round (reconnect or next ping). - If the DO crashes mid-
compact(): thetransactionSyncensures atomicity. Worst case is a snapshot row with overlapping ops in the log; replay is idempotent.
Log this category as soft-fail: structured log entry { kind: 'soft_fail', phase: 'mid-apply'|'mid-append'|'mid-broadcast'|'mid-compact', wsId, ts }. Don't surface to the user. Don't throw.
7.2 Client clock skew
Yjs uses Lamport-style logical clocks per client (clientID + monotonic seq), not wall clocks. Skew is invisible at the protocol layer. ts in the ops log is server-side Date.now(), so the broker never trusts client wall clocks. No mitigation needed. Documented here so reviewers know why we don't validate client timestamps.
7.3 Concurrent same-field edits
Yjs converges deterministically:
Y.Map<string, primitive>field set by two clients concurrently: the update with the higher (clientID, clock) tuple wins, applied identically on every replica. No data loss in the sense of inconsistency; one user's value is overwritten by another's, exactly as a single-user LWW would behave for that field.Y.Textconcurrent edits at the same position: character-level merge. Both contributions are preserved.Y.Arrayconcurrent inserts at the same index: causal ordering produces a deterministic order on every replica.
The broker does NOT need to mediate. We document the property here so the implementer knows not to add "conflict resolution" logic — Yjs has it.
7.4 Long disconnect → catch-up payload too large
If a client has been offline for a long time and the server's "diff since your state vector" exceeds a hard cap, refuse the incremental delta and send a full state snapshot instead.
const CATCHUP_CAP_BYTES = 2_000_000; // 2 MB
const diff = Y.encodeStateAsUpdate(this.doc, clientStateVector);
if (diff.byteLength > CATCHUP_CAP_BYTES) {
// Send full snapshot (smaller in pathological history-heavy cases) and
// a fresh empty state vector, OR simply send the full snapshot if it's smaller.
const full = Y.encodeStateAsUpdate(this.doc);
send(sock, syncStep2(full.byteLength < diff.byteLength ? full : diff));
} else {
send(sock, syncStep2(diff));
}
The 2 MB cap is conservative; on Workers we have plenty of headroom but the client's IndexedDB / memory budget is the real constraint. Open question §11.4.
7.5 Malformed op from client
Per principle #3 (no fallbacks), the broker validates EVERY incoming Yjs update against lib/schemas Zod definitions BEFORE applying. Validation strategy in §8.
On validation failure:
- Do NOT apply the update.
- Do NOT broadcast.
- Do NOT log to ops table.
- Send a
messageAuth-style deny to the offender? No. Yjs auth messages are for permission denial, which doesn't apply here. Instead, define a new envelope:
// messageType = 100 (custom; out of y-protocols range)
// payload: { code: 'schema_invalid', issues: ZodIssue[], updateClock: clientClock }
The client logs and surfaces. Repeated violations from one socket → close socket with code 1008 (policy violation). Threshold: 5 violations within 60s.
A successfully validated but semantically suspect update (e.g., creates a section with a kind field not in the registry) — the schema should catch that at parse time. If the schema doesn't catch it, that's a lib/schemas bug, not a broker concern.
7.6 D1 unavailable during cold backup
Log { kind: 'd1_backup_failed', wsId, error } and re-arm the alarm with backoff (next attempt: 5min, 15min, 60min). DO SQLite is unaffected — the DO continues serving live traffic. Backup is best-effort.
8. Schema validation at ingest
Every Yjs update is validated against lib/schemas before being applied to the canonical Y.Doc. Reject invalid ops without applying. Principle #3.
8.1 The challenge
A Yjs update is opaque binary — it doesn't directly say "set jobs[abc].title = 'X'". To validate semantics we have to:
- Decode the update against a temp
Y.Docsnapshotted from current state. - Diff the resulting state against current state to extract the logical changes (which paths in the doc tree changed, what the new values are).
- Run each change through the corresponding Zod schema.
- If all changes validate, apply the original update bytes to the real
this.doc. If not, reject.
8.2 Implementation sketch
async validate(update: Uint8Array): Promise<ValidationResult> {
const probe = new Y.Doc();
Y.applyUpdate(probe, Y.encodeStateAsUpdate(this.doc)); // clone current
Y.applyUpdate(probe, update); // apply candidate
const changes = diffYDocs(this.doc, probe); // returns Path[]
for (const change of changes) {
const schema = pickSchema(change.path); // routes to lib/schemas
const parsed = schema.safeParse(change.newValue);
if (!parsed.success) return { ok: false, issues: parsed.error.issues, path: change.path };
}
return { ok: true };
}
pickSchema(path) is a dispatcher in lib/mutations that maps paths like ['jobs', jobId] → JobSchema, ['settings'] → WorkspaceSettingsSchema, etc.
8.3 Performance
Cloning the Y.Doc on every update is the obvious cost. Mitigations:
- For small workspaces (~95% of cases) the clone+diff costs <1ms.
- For large workspaces, switch to delta-decoding the update via
Y.decodeUpdateand walking only the touched structures. Optimization deferred until measured.
Open question §11.2.
8.4 What schemas exist
Validated paths (each maps to a Zod schema in lib/schemas):
| Path prefix | Schema |
|---|---|
settings |
WorkspaceSettingsSchema |
layout |
LayoutSchema |
order_state |
OrderStateSchema |
cv_profile.<lang> |
LangTreeSchema (recursive Node tree under tree) |
cl_profile.<lang> |
LangTreeSchema (recursive Node tree under tree) |
jobs.<jobId> |
JobSchema |
jobs.<jobId>.tailored_cv |
NodeSchema (full Node-tree snapshot) |
jobs.<jobId>.tailored_cl |
NodeSchema (full Node-tree snapshot) |
jobs.<jobId>.evaluations |
EvaluationsMapSchema |
jobs.<jobId>.origins |
CellOriginsMapSchema |
cloud_keys |
CloudKeysSchema |
If pickSchema returns no schema for a path, that's a bug — throw, don't pass. Principle #3.
9. Ops the broker accepts
The broker doesn't have RPC-shaped ops; it accepts Yjs updates that touch any of the documented paths. At a high level the catalog is:
- Workspace settings:
settings.*,layout.*,order_state.*. - Profiles:
cv_profile.<lang>.tree(recursive Node tree withvariant-poolwrappers),cl_profile.<lang>.tree. Plusformatper language. - Jobs lifecycle: create (
jobs.<id> = {...}), update (any sub-path), delete (jobs.<id>removed). - Per-job tailored output:
jobs.<id>.tailored_cv,jobs.<id>.tailored_cl(full Node-tree snapshots). - Per-job derived data:
jobs.<id>.evaluations,jobs.<id>.origins. - Cloud keys / cascade chains:
cloud_keys.providers,cloud_keys.chains.
The typed op catalog that produces these updates is in lib/mutations/src/opsCatalog.ts; current OP_CATALOG_VERSION is exported from lib/mutations/src/index.ts. See CLAUDE.md §24 for the canonical op contract.
Awareness (cursor positions, presence) is separate — see §4.2.
This is descriptive, not prescriptive: any Yjs update touching the documented Y.Doc shape is valid input. Schemas in lib/schemas define what's accepted under each path; the broker is path-agnostic beyond schema dispatch.
10. Hibernation costs
10.1 Wake-up budget
| Phase | Estimate |
|---|---|
| DO cold start (Cloudflare runtime instantiation) | 30-100 ms |
| Constructor (restore session map) | <5 ms |
hydrate() SQLite reads (snapshot + ops log) |
5-30 ms typical, up to 100 ms with full N=200 op log |
Y.applyUpdate of snapshot |
<5 ms typical |
| Replay of ops log (≤200 small updates) | 5-50 ms |
| Total p95 | ~150 ms |
This is well within the user-perceived "instant" budget for a reconnect.
10.2 Behavior during wake
While hydrate() is in-flight:
webSocketMessagehandler enqueues incoming frames into a per-socketpendingFrames: Uint8Array[].- Per-socket queue capped at 32 frames; on overflow, close the socket with code 1011 (server overload). Client reconnects, re-syncs. Better than memory-bombing the DO.
- After
hydrate()resolves, drain the queues in arrival order per socket. Awareness frames are processed first within each queue (cheap, no validation), then sync frames.
10.3 What we explicitly don't do
- Don't pre-warm. Cloudflare runtime decides; we cooperate.
- Don't snapshot-on-hibernate. Hibernation is silent; we can't run code in the hibernate path. The M=5min timer plus the alarm-fired backup pattern covers durability.
11. Open questions
These need user input before implementation. Default behavior is listed; flag them at decision time.
Snapshot frequency tradeoff (N, M). Current proposal: N=200 ops, M=5min. Higher N means longer wake replay but fewer compaction events; lower N means faster wake but more SQLite churn. Want to gut-check with realistic editing telemetry before locking. Default: ship with N=200, M=300s, instrument and revisit.
Schema validation cost on large workspaces. Cloning the full Y.Doc per incoming update is O(state size). For workspaces with many jobs (hundreds), this could exceed the 10ms-per-request principle. Alternative: parse the update bytes directly and validate just the touched subtree. More complex; harder to be sure we caught everything. Default: ship with full-clone, measure, optimize if needed.
D1 backup cadence. Current proposal: 1 hour while active. Should we also force a backup right before announced maintenance windows or DO migrations? Default: hourly only; rely on operational discipline.
Catch-up payload cap (CATCHUP_CAP_BYTES). 2 MB is a guess. Real constraint is client IndexedDB write speed + initial render budget. Default: ship 2 MB, watch for "long disconnect" telemetry.
R2 archival of old snapshots.
ARCHITECTURE.mdmentions R2 holds older snapshots. This design doesn't specify the archival job. Open: is it a cron Worker that scans D1 for oldworkspace_sub_docrows, or does the DO itself push to R2 when a new snapshot supersedes the old? The latter couples the DO to R2 writes (more failure surface). Default: out of scope; design separately when archival becomes necessary.
Appendix A — Type sketches
// relay/cf/src/workspace-doc.ts — target shape
import * as Y from 'yjs';
import { Awareness, encodeAwarenessUpdate, applyAwarenessUpdate } from 'y-protocols/awareness';
import * as syncProtocol from 'y-protocols/sync';
import * as encoding from 'lib0/encoding';
import * as decoding from 'lib0/decoding';
import { validateUpdate } from '@cv/mutations'; // §8
const MSG_SYNC = 0;
const MSG_AWARENESS = 1;
const MSG_QUERY_AWARENESS = 3;
const MSG_SCHEMA_REJECT = 100; // §7.5
type Session = { userId: string; connectedAt: number };
interface Env {
CV_DB: D1Database;
}
export class WorkspaceDoc {
private state: DurableObjectState;
private env: Env;
private sql: SqlStorage;
private doc = new Y.Doc();
private awareness = new Awareness(this.doc);
private sessions = new Map<WebSocket, Session>();
private hydrated = false;
private opCounter = 0;
private opsSinceSnapshot = 0;
private lastSnapshotTs = 0;
private violationsBySocket = new WeakMap<WebSocket, number[]>();
constructor(state: DurableObjectState, env: Env) {
this.state = state;
this.env = env;
this.sql = state.storage.sql;
this.bootstrapSchema();
for (const ws of state.getWebSockets()) {
const meta = ws.deserializeAttachment() as Session | null;
if (meta) this.sessions.set(ws, meta);
}
}
private bootstrapSchema(): void { /* CREATE TABLE IF NOT EXISTS … */ }
private async hydrate(): Promise<void> { /* §1.2 */ }
async fetch(req: Request): Promise<Response> {
const upgrade = req.headers.get('Upgrade');
if (upgrade !== 'websocket') return new Response('expected websocket', { status: 426 });
await this.hydrate();
const url = new URL(req.url);
const userId = url.searchParams.get('userId') ?? `anon_${crypto.randomUUID()}`;
const pair = new WebSocketPair();
const [client, server] = [pair[0], pair[1]];
this.state.acceptWebSocket(server);
const session: Session = { userId, connectedAt: Date.now() };
server.serializeAttachment(session);
this.sessions.set(server, session);
// Server-initiated sync step 1
this.sendSyncStep1(server);
// Send current awareness snapshot
this.sendAwarenessSnapshot(server);
return new Response(null, { status: 101, webSocket: client });
}
async webSocketMessage(ws: WebSocket, msg: ArrayBuffer | string): Promise<void> {
if (typeof msg === 'string') return; // we don't speak JSON anymore
await this.hydrate();
const decoder = decoding.createDecoder(new Uint8Array(msg));
const messageType = decoding.readVarUint(decoder);
switch (messageType) {
case MSG_SYNC: return this.onSync(ws, decoder);
case MSG_AWARENESS: return this.onAwareness(ws, decoder);
case MSG_QUERY_AWARENESS: return this.sendAwarenessSnapshot(ws);
default: /* unknown — ignore */ return;
}
}
async webSocketClose(ws: WebSocket): Promise<void> {
const session = this.sessions.get(ws);
this.sessions.delete(ws);
if (session) {
// Remove this client's awareness state and broadcast removal
const clientID = (this.awareness as any).clientIDsBySocket?.get(ws); // tracked at connect
if (clientID != null) {
this.awareness.removeAwarenessStates([clientID], ws);
}
}
}
async alarm(): Promise<void> {
await this.hydrate();
await this.maybeCompact(/* force */ true);
await this.backupToD1();
if (this.opsSinceSnapshot > 0 || /* new ops since backup */ false) {
this.state.storage.setAlarm(Date.now() + 60 * 60 * 1000);
}
}
// — internal —
private onSync(ws: WebSocket, decoder: decoding.Decoder): void { /* §3, §4.1, §8 */ }
private onAwareness(ws: WebSocket, decoder: decoding.Decoder): void { /* §4.2 */ }
private sendSyncStep1(ws: WebSocket): void { /* §3.2 */ }
private sendAwarenessSnapshot(ws: WebSocket): void { /* §4.2 */ }
private appendOp(update: Uint8Array, origin: string): void { /* §2.2 */ }
private async maybeCompact(force?: boolean): Promise<void> { /* §5 */ }
private async backupToD1(): Promise<void> { /* §6 */ }
private rejectUpdate(ws: WebSocket, issues: unknown, clock: number): void { /* §7.5 */ }
private broadcastUpdate(update: Uint8Array, except: WebSocket): void { /* §4.1 */ }
}
Appendix B — Differences from v1 broker
| Concern | v1 (workers/realtime) |
v2 (this design) |
|---|---|---|
| Source of truth | Clients (DO is dumb relay) | DO (canonical Y.Doc) |
| Persistence | None | DO SQLite (snapshot + ops) + D1 hourly backup |
| Wire format | Binary blind-relayed + JSON envelopes for presence | Yjs sync protocol + Yjs awareness, all binary |
| Validation | None | Zod schema per update before apply |
| Multi-tab sync | Best-effort, broken in BUG-2 | CRDT-correct |
| Hibernation safety | Sessions survive; doc state has nowhere to live | Doc state durable in DO SQLite |
| Sync after disconnect | Clients full-refetch via D1 | Yjs vector clock delta sync |
| Presence | JSON { type: 'presence', count, ... } |
Yjs Awareness, same UI surface |
| Zombie gate | D1 existence check before idFromName |
Same — keep verbatim |