Missing Tests Handoff

This is a focused implementation brief for Claude Code. The goal is to turn classified coverage debt into executable tests without weakening the gate.

Current audit command:

bun scripts/audit-story-coverage.ts --json --allow-known-product-bugs

Current state:

  • Catalog rows: 151
  • Green executable rows: 114
  • Critical/High rows: 113
  • Critical/High non-green rows accepted as debt: 14
  • Missing rows: 0
  • Default audit exits PASS_WITH_DEBT. There are no current coverage-status: known product bug story specs.

Important distinction:

  • missing: 0 only means every story is traceable.
  • It does not mean every story is tested.
  • fixme, debt, and manual rows are not green.

What Was Missed

Claude's scaffold made the catalog traceable, but it did not finish executable coverage. The largest misses are:

  • Placeholder specs with test.fixme(...) instead of real browser assertions.
  • Waivers for Critical/High stories that need harness work.
  • No real P2P/WebRTC browser sync coverage. lib/workspace-client/src/transport/p2p.ts and lib/workspace-client/src/transport/ws.ts are still stubs.
  • Several multi-browser stories prove durable reload/poll convergence, not live realtime fanout.
  • Efficiency/system stories are listed as waivers rather than enforced checks.

Do not make the audit green by deleting story IDs, relaxing severity, adding waivers, widening timeouts without proof, or moving failures out of the gate.

Product Bugs Resolved In This Pass

These were previously hard red rows. They now have executable regression coverage and no coverage-status: known product bug marker.

ID Severity File Required regression
P2-G10-S3 Critical ui/e2e/stories/collab/p2-g10-s3-both-offline-reconnect.spec.ts Two actors edit offline, reconnect, and both see the merged state.
P2-G6-S2 Critical ui/e2e/stories/collab/p2-g6-s2-same-bullet-char-merge.spec.ts Two actors type into the same CVL bullet and character-level CRDT merge survives.
P4-G6-S6 Critical ui/e2e/stories/agent/p4-g6-s6-fixed-tier-llm-protection.spec.ts Non-human node.update cannot mutate FIXED-tier nodes, even with force: true.
P3-G3-S1 High ui/e2e/stories/collab/p3-g3-s1-mentor-disagrees-knockouts.spec.ts Mentor and primary can disagree on independent knockout fields without clobbering.
P3-G6-S3 High ui/e2e/stories/collab/p3-g6-s3-mentor-edits-while-primary-tailors.spec.ts Tailored snapshot reflects the state at tailor time while concurrent mentor edits are preserved.
ADV-S2 Adversarial ui/e2e/stories/adversarial/html-in-description.spec.ts Script/HTML payloads in job descriptions do not execute in the browser.
P7-G10-S2 Medium ui/e2e/stories/device/workspace-handoff.bob-rejoins-without-duplicate-identity.spec.ts Reopening a workspace in the same browser reuses the workspace-scoped device identity.

High-Priority Missing Executable Tests

Implement these first. Each should become a real spec with no test.fixme(...) and no coverage-status: needs harness.

ID Severity Current state Required implementation
P6-G5-S1 Critical waiver / needs harness Two browser contexts. Laptop drags kanban; second context reflects within 2 s. Use existing drag helpers if possible.
ADV-S9 High fixme / needs harness Seed or corrupt one sub-doc, open workspace, assert a graceful error surface and prove healthy sub-docs still load and remain writable.

System And Efficiency Coverage

These are not normal user stories; implement them as fast CI checks, script assertions, or focused integration tests.

ID Current state Required implementation
S2 waiver / needs harness Provider failure normalization matrix. Assert all provider failure shapes normalize to the expected product error envelope.
E2 waiver / needs harness Enforce fast-check duration budget in CI.
E3 waiver / needs harness Enforce repeatability/flakiness budget for slow free checks.
E4 waiver / needs harness Prove browser smoke runs without paid-provider spend. This should consume the paid-forbidden sentinel, not duplicate it.
E5 waiver / needs harness Release gate starts from zero-budget evidence. Add a sentinel that fails if a release gate requires live paid providers.
E7 waiver / needs harness Synthetic test data cleanup budget: assert count/time ceilings for cleanup.

Leave these gray unless the quality service is now in scope:

  • S5: Quality service is a product surface.
  • S6: Quality results are auditable.
  • E6: Quality service rejects ambiguous spend.

Cross-Cutting Harness Gaps

These are important but need deterministic hooks. Add the hooks in small, explicit helpers rather than encoding timing races.

ID Current state Required implementation
X-G2-S1 waiver / needs harness Three actors: user pastes URL, agent extracts in background, friend opens detail, all see final extracted fields.
X-G6-S1 waiver / needs harness Tailor triggered while friend edits source bullet; snapshot reflects tailor-time state. Needs deterministic tailor delay hook.
X-G6-S2 waiver / needs harness Agent tailors while user offline; reconnect preserves in-flight local edits and shows agent snapshot. Needs deterministic offline simulator.
X-G10-S1 waiver / needs harness Workspace settings save and job order save in same gesture; one atomic batch, peers see consistent state.

P2P / Realtime Boundary

Do not claim P2P/WebRTC is covered yet.

Current proof only covers pieces:

  • Multi-browser durable convergence exists.
  • P2P signaling smoke exists.
  • WebRTC/P2P transport implementation is still stubbed.

If P2P is in scope, implement and test:

  1. Real P2pTransport datachannel/Yjs exchange between two browser contexts.
  2. A browser spec that forces the P2P path, edits a CVL Y.Text in both browsers, and asserts both converge without relying on reload.
  3. A fallback spec where P2P fails and the app falls back to server relay / REST without data loss.

If P2P is not in this cutover scope, document it as gray/future or yellow debt. Do not mark it green.

Definition Of Done

After implementing missing tests:

bun scripts/audit-story-coverage.ts
bun scripts/check-paid-forbidden.ts
bun run check
bun run test

Expected result now:

  • Default audit exits PASS_WITH_DEBT.
  • Hard failures stay at zero.
  • Do not increase red/yellow/blue counts unless a new story was deliberately added and classified.

Final handoff should list:

  • Which story IDs were made green.
  • Which helper/harness pieces were added.
  • Which stories remain yellow/blue/gray and why.
  • Exact commands run and their results.
Source: wiki/content/stories/MISSING-TESTS-HANDOFF.md