CRDT-pattern gotchas

A short field guide for anyone writing or reviewing Yjs ops in CareerVector. Each entry pairs a tempting-looking pattern with a counter-example that bit us, and the canonical fix that survived review.

Companion to REALTIME-ARCHITECTURE-V3.md and REALTIME-DESIGN.md. Those cover the multi-pathway resilience model and the thin-relay broker contract; this document covers the operational CRDT semantics you have to understand to write correct ops.

1. Y.Map.set is a REPLACE, not a merge

If you write a brand-new sub-map at an existing key, peers do not field- merge — the higher-clientID entry wins on .get() and the loser's fields disappear. Two clients that independently jobs.set('7', new Y.Map()) (e.g. during cold-load) will lose data.

Anti-pattern

// Each client mints its own Y.Map for the same key — last writer by clientID
// wins; the loser's per-field edits become invisible.
const job = new Y.Map();
for (const [k, v] of Object.entries(rowFromD1)) job.set(k, v);
jobs.set(jobId, job);

Fix

• For seeding (cold-load), only run when the entry does not already exist (if (!jobs.has(id)) ...).
• For updates on an existing entry, mutate the existing sub-map: jobs.get(id).set(field, value) — same item identity, field-level merge.
• For server-originated whole-entry replacements, accept the clobber and use a high-priority clientID so the replacement always wins (see §3).

2. Y.Array.delete + push does not survive cold-load

Treating reorders as delete(idx) + push(item) produces an update whose correctness depends on the array's prior item identities. Cold-load that re-creates the array fresh (different item identities) replays as delete(idx) against an array of different shape — the wrong item is removed or replay throws. This is the section-reorder bug (commit cff90977).

Fix: structural changes go through whole-tree replacement. branchRoot.set('tree', nodeToYMap(...)) — atomic, commutative, and identical to what cold-load produces.

3. Server-originated ops in thin-relay need a stable, high-priority clientID

In Phase 4 thin-relay, the DO holds no authoritative Y.Doc, so the server has no authoritative item identity. Naïvely broadcasting a server-side job.update from a transient Y.Doc with a random clientID:

• Bloats the SV by one slot per REST call (one stale clientID per op).
• Loses the LWW tie when the recipient already has a higher random clientID.

Fix (in relay/cf/src/workspace-doc.ts:stableClientId):

• Stable per-workspace clientID derived via fnv-1a — single SV slot for the "server actor".
• Top of the uint32 range (0xFFFE_0000 | (hash & 0xffff)) so it always beats any per-browser random clientID in Yjs's last-write-wins-by-clientID tie-breaker.
• Whole-entry replacement semantics: job.update is promoted to a synthetic job.create carrying the post-update workspace job. Race window with concurrent client edits is documented and bounded.

4. Y.encodeStateAsUpdate is NOT canonical

Two Y.Docs holding the logically identical state can produce different binary updates (different item ordering, different clientID metadata). For equality checks use a JSON projection (doc.toJSON()); for "did anything change?" diffs use the state vector and encodeStateAsUpdate(doc, sv).

5. Cold-load gates must be PER-SUBTREE

Using one subtree's emptiness as a proxy for "everything is hydrated" breaks the moment a different subtree is mutated server-side. The forms-validation regression was exactly this: cv_profile.size > 0 was treated as "IDB is complete," dropping the jobs fetch on every reload — even though jobs is a wholly independent Y.Map root.

Fix: cold-load merges run unconditionally. Each subtree decides internally whether its data is safe to merge:

6. Snapshots are written by the leader tab snapshot writer

Phase 4's POST /snapshot/:wsId is the only way the D1 y_snapshot column moves forward. The SnapshotWriter class (snapshot-writer.ts) closes this loop: the leader tab periodically posts Y.encodeStateAsUpdate(doc) to the endpoint on a 30-second cadence (while dirty) and on visibilitychange:hidden / beforeunload (via navigator.sendBeacon). After the first flush, cold-load via /snapshot/:wsId returns 200 with the full Y.Doc state, bypassing the slower coldLoadFromRest fallback.

See §13 for the full design and dirty-flag origin classification.

7. Origin matters for doc.on('update', ...)

If you transact(..., origin) with one symbol but listen for a different origin, your handler sees nothing — and forgetting to filter origins makes seed steps re-broadcast (the thin-relay handler trips over this if you do not split origins).

Pattern for "capture op bytes but not seed bytes":

const opOrigin = Symbol('op');
const seedOrigin = Symbol('seed');
const handler = (update, origin) => {
    if (origin === opOrigin) opBytes = update;
};
doc.on('update', handler);
doc.transact(seedDoc, seedOrigin);   // seed bytes ignored
doc.transact(applyOp,  opOrigin);    // op bytes captured
doc.off('update', handler);

8. Flat-map seeding paradox in thin-relay

When a server-side op patches an existing Y.Map (e.g. settings.update), it is tempting to first seed the prior values into the transient Y.Doc so the broadcast looks like a minimal delta. Do not do this.

A seed step inserts Yjs items into the doc. The patch op then creates items whose left/origin reference the seed items' (clientID, clock) pair. A connected peer (which has a different Y.Doc with different item identities) receives the broadcast and cannot resolve those references — the update lands in "pending integrations" and is silently dropped.

Fix: Apply flat-map patches directly to a completely empty transient Y.Doc. The resulting items have no origin references; any peer integrates them unconditionally via LWW. The trade-off is that the broadcast carries only the patched fields, not the prior state — but for LWW flat-map semantics that is correct.

9. Tree-path delta unresolvability in thin-relay

For tree-path ops (node.*, node.batch, selection.update, etc.), the server must seed the profile tree from D1 before applying the op (so applyOp can traverse the tree). But the delta bytes reference the seeded items' identities — which differ from the connected peer's tree items (attributed to the peer's own clientID from cold-load). The peer cannot resolve those references and the update is dropped.

Fix (whole-tree-replace promotion): After applying the tree-path op to the seeded transient doc, extract the mutated tree via treeMap.toJSON(), then build a fresh Y.Doc and apply a synthetic cv_profile.replace (or cl_profile.replace). The resulting update bytes are self-contained — no foreign item references — and any peer integrates them as LWW on the lang key. This is the same CRDT pattern as job.update -> job.create.

10. Stable clientID clock collision for sequential thin-relay ops

Each thin-relay request builds a fresh transient Y.Doc with the stable clientID (§3). Each fresh doc resets the internal clock to 0. Two sequential requests produce two independent frames both starting at clock 0 under the same clientID. When a peer applies frame A then frame B, both at (X, 0), Yjs treats B's items as duplicates of A's items and silently skips the second op.

This means a peer that applies job.create (frame 1) then settings.update (frame 2) within the same session will not see the settings change — the settings items clock-collide with the job items.

Current mitigation: Each broadcast frame is independently correct when applied to a fresh peer (which has no prior state for either frame). The collision only manifests when a peer attempts to merge two server-originated frames in a single session. In practice this race window is small (REST writes happen asynchronously with irregular timing), and each individual op lands correctly on a peer that was connected at the time of the broadcast.

Full fix (out of scope for Phase 4): Persist the server's Yjs clock across requests (DO durable state or a D1 counter). Then each thin-relay request can start from the last-known clock, eliminating collision.

11. Cold-load schema completeness

Every D1 column that any component reads from the Y.Doc jobs map must be returned by GET /api/workspaces/:id and seeded into the Y.Doc by coldLoadFromRest(). Hardcoded {} or null or simply omitting a field is a silent regression: the component sees undefined on cold-load and silently falls back, misbehaves, or displays stale state until a Yjs peer joins.

Pattern for auditing: Compare the JobSeedRow interface and the SQL SELECT in +server.ts against every field accessed from jobsStore.value in components and pipeline.svelte.ts. Any field present in the latter but absent from the former is a gap.

Known regressions and fixes (COLD-LOAD-AUDIT-2026-04-28):

What does NOT need to be in the cold-load payload:

• format_overrides — no component reads from Y.Doc; CvGenerator fetches fresh via individual job endpoint.
• raw_scrape — pipeline-internal write-only field, never displayed.
• location_id — server-side join key used by the commute API only.

12. Compute lock TTL must match the task's worst-case LLM duration

Symptom: Two browser tabs both run the same LLM stage for the same job — BYOK key charged twice, conflicting results written to Yjs.

Root cause: The compute_lock table uses a TTL-based expiry. If the TTL is shorter than the actual LLM call, the cleanup sweep that runs on every acquireLock call deletes a still-active lock and lets a second tab steal it.

The tailor stage (CV + CL with 4-concurrent section calls, up to 2 critic-loop iterations, and 2 selection retries per section) routinely takes 90–120 s on slow providers. The old uniform 60 s TTL was too short.

Anti-pattern (uniform 60 s for all tasks):

T=0    Tab A acquires lock for 'tailor'  (TTL=60)
T=90   Tab B calls acquireLock
         → cleanup: locked_at (T=0) < now - 60s (T=30) → DELETE Tab A's lock ✓
         → INSERT → Tab B wins
T=90   Tab A is still running its LLM call
       Both tabs will persist conflicting tailored documents.

Fix — per-task TTL (Option C):

lib/domain/src/lock.ts exports TASK_TTL:

Callers pass TASK_TTL[task] to acquireLock() and include it as ttl_seconds in the POST /api/.../compute-lock body. The server endpoint validates and applies the TTL to the DELETE sweep, then passes it to acquireLock().

Blast-radius trade-off: A crashed tab holding a tailor lock will block new acquires for up to 180 s (vs 60 s before). This is acceptable: tailoring is a one-time per-job event and the next tab can wait. Faster tasks (salary: 30 s) are now tighter than the old uniform TTL, so stale salary locks clear faster.

Tests: lib/domain/tests/lock.test.ts — "TASK_TTL — per-task TTL prevents premature lock expiry" suite, including a regression test that explicitly demonstrates the old 60 s TTL would have stolen the tailor lock at T=90 s.

13. Persistence: leader tab snapshot writer

All Yjs ops MUST eventually reach D1 even if no peer ever issues a REST write. The leader tab (per L0 leader election) writes Y.encodeStateAsUpdate(doc) to POST /api/workspaces/:id/snapshot on cadence (~30s while dirty), visibilitychange:hidden, and beforeunload. Use navigator.sendBeacon for the unload triggers — fire-and-forget guaranteed delivery. Followers detect dirty but don't duplicate POSTs.

Dirty-flag origin classification:

Clock header: X-Snapshot-Clock = sum of all client clock entries in the state vector. The server's conditional write (y_snapshot_clock < incoming) prevents stale snapshots from overwriting newer ones in the concurrent-leader-flush race. For sendBeacon (no custom headers), the clock is passed as ?clock=N in the URL.

Implementation: ui/src/lib/realtime/snapshot-writer.ts — SnapshotWriter class. Integration: ui/src/lib/realtime/connection-impl.ts — instantiated in WorkspaceConnection.connect(). Tests: ui/src/lib/realtime/snapshot-writer.test.ts. Endpoint: ui/src/routes/api/workspaces/[id]/snapshot/+server.ts (POST). E2E: ui/e2e/persistence-snapshot.spec.ts.

Snapshot writes happen client-side via the leader-tab SnapshotWriter; the broker no longer needs a write path for snapshots beyond accepting the bytes.

Reference implementations