Wave 3 Report: Dev/Test Reliability + CVL Reorder Regression

Agent: sonnet-devtest-reliability Date: 2026-04-25 Worktree: /home/richc/Documents/GitHub/careervector-wave3-sonnet-devtest-reliability Branch: wave3-sonnet-devtest-reliability Base: b9ee2d1 (Fix CVL reorder preview bugs and add e2e harness) Commit: 15a88a6

Summary

Three bounded reliability improvements delivered, all within scope. Build and full test suite green. 5/5 e2e tests pass.

Files Changed

git diff --stat: 6 files, 102 insertions, 13 deletions.

Deliverables

1. Stable test hooks for drag grips (SD-1)

Change: DragGrip in src/components/ui/SortableList.tsx now has data-testid="section-drag-grip". The section toggle button in src/components/ui/Section.tsx now has data-testid="section-toggle".

e2e update: sectionGripAt(page, index) in e2e/helpers/cvl.ts uses:

page.locator(`[data-index="${index}"] [data-testid="section-drag-grip"]`)

instead of the brittle button.cursor-grab CSS-class selector.

Two new helpers added:

• toggleSection(page, index) — click the expand/collapse button for a section
• isSectionExpanded(page, index) — point-in-time snapshot; for test assertions use the span.rotate-90 locator with await expect(locator).toBeVisible() directly (retry-based, see below)

2. Playwright regression for BUG-3 ("expanded section stays expanded after reorder")

New test: CVL section reorder — expand state › expanded section stays expanded after drag reorder

Location: e2e/cvl-reorder.spec.ts:123

What it covers:

• Expands a section (index 2 = summary-tech, same "Summary" label group as index 1 = summary)
• Drags it one position up (index 2 → index 1)
• This triggers onSlugReorder with a disambiguation rename (two adjacent "Summary" sections swap positions)
• Asserts the section is still expanded at its new position (index 1) after the drag and PUT save

Why drag 2→1: Both sections have label "Summary", so the rename triggers the idMap path in handleDragEnd. Before the BUG-3 fix (setOpenSections(remap)), the expanded state tracked the old ID and the section would silently collapse. This test would fail on pre-b9ee2d1 code.

Assertion pattern used:

await expect(
    page.locator('[data-index="1"] span.rotate-90'),
    'Section must remain expanded after reorder',
).toBeVisible({ timeout: 5_000 });

await expect(locator).toBeVisible() retries until the DOM settles, which is correct after a React re-render triggered by drag. A point-in-time isVisible() call immediately after savePromise is unreliable because React's state flush may lag the network response.

3. Fresh local D1 bootstrap

Decision: Add db:bootstrap script that runs schema.sql only. Do NOT rename existing migration files.

Why schema.sql only: schema.sql is the complete current schema and already contains the connections column and telemetry_events table. Running add-connections-column.sql on a fresh DB fails with "duplicate column name: connections". The incremental migrations are for upgrading existing databases that predate those changes.

Script added:

"db:bootstrap": "wrangler d1 execute careervector --local --file=migrations/schema.sql"

Usage:

# Fresh local setup (replaces broken db:migrate on a clean clone):
npm run db:bootstrap

# Incremental upgrade on existing DB (unchanged):
npm run db:migrate

Verified: npm run db:bootstrap ran successfully on a clean worktree (19 commands in schema.sql, all succeeded). The local D1 was immediately usable by npm run dev and all e2e tests.

Important constraint: Do NOT run db:migrate after db:bootstrap. Wrangler's migrations apply tracks applied migrations in d1_migrations. After db:bootstrap (which uses execute, not migrations apply), db:migrate would try to apply all migrations alphabetically and fail on "duplicate column name". Bootstrap and migrate are mutually exclusive for a given local DB.

Docs updated: playwright.config.ts now documents npm run db:bootstrap as the correct setup command, replacing the broken four-step manual sequence.

Validation

All commands run in the worktree. node_modules symlinked from the main worktree (the worktree does not have its own install).

npx astro sync + npx tsc --noEmit

✓ astro sync: types generated in 121ms
✓ tsc: only pre-existing infra/ Pulumi errors (4 errors, @pulumi/* not installed locally)
  All src/, tests/, and e2e/ files type-check cleanly.

npm run build

✓ Server built in 6.50s
✓ Complete

npm run test:unit

Test Files  50 passed (50)
     Tests  892 passed (892)

No regressions. The unit test count is unchanged from wave-2 (892).

npm run test:e2e

Dev server started via npm run dev (wrangler local D1 bootstrapped with npm run db:bootstrap).

Running 5 tests using 1 worker

  ✓  1 CVL section reorder — persist › reordering two sections persists after page refresh (2.2s)
  ✓  2 CVL section reorder — expand state › expanded section stays expanded after drag reorder (1.8s)
  ✓  3 CVL section reorder — preview renders › preview renders at least one page on initial load (2.6s)
  ✓  4 CVL section reorder — preview renders › preview re-renders after section reorder without blank pages (6.5s)
  ✓  5 CVL doc switch — no stale state › switching between CV and CL clears the section list (673ms)

5 passed (14.3s)

npm run test:e2e:known-bugs

  ✘  1 CVL two-tab sync [KNOWN BUG] › Tab B sees section reorder from Tab A via realtime (5.1s)
1 passed (5.5s)   ← expected-fail annotation, not a real failure

Residual Risks

1. isSectionExpanded() is point-in-time. The helper returns await locator.isVisible() which is a snapshot. It's fine for checking state that has already settled (e.g., after toggle click + waitForSelector). It should NOT be used directly in assertions after async operations (drag, save) — use await expect(locator).toBeVisible() instead. The helper is still useful for conditional logic and is documented accordingly.

2. db:bootstrap and db:migrate are mutually exclusive. Running both on the same local DB will fail. This is documented in playwright.config.ts and this report. If someone runs both, they need to delete .wrangler/state/v3/d1/ and re-bootstrap. A more robust solution (numeric migration prefixes) was not implemented because renaming existing migration files risks breaking remote D1 migration history.

3. data-testid on DragGrip not namespaced. The testid is section-drag-grip (not section-drag-grip-{id} or section-drag-grip-{index}). Since tests already scope by [data-index="N"], this is unambiguous. If DragGrip is ever used outside the CVL section list (e.g., in a header editor), a more specific testid would be needed.

4. span.rotate-90 as expand-state signal. This relies on Tailwind JIT not purging rotate-90. The class IS present in source via ${isOpen ? 'rotate-90' : ''}, so JIT will keep it. If Section.tsx is refactored to use a different CSS approach, the e2e expand-state tests would silently lose their assertions. A data-testid="section-body" on the expanded content div would be more robust but was out of scope.

5. Persist test flakiness at far drag distances. The original persist test uses dragSectionToSection(page, summaryBefore, experienceBefore) = drag from index 1 to index 4. This worked in wave-2 and in repeated runs here but failed once in the first run of this session. Cause unknown — possibly a timing issue with the virtual list's absolute positioning and dnd-kit's collision detection during fast programmatic drags. The expand-state regression test avoids this by using index 2→1 (adjacent drag, proven reliable).

Recommendation

Merge.

All three deliverables are narrow, safe, and validated:

• Source changes: 2 single-line data-testid additions in production UI code (zero behavioral change)
• Test changes: 1 new regression test (correct, passing, covers BUG-3 scenario) + selector hardening
• Script: db:bootstrap is a safe addition (execute not migrations apply, does not touch prod state)
• No schema changes, no wrangler.toml changes, no dependency changes

Merge after opus-cvl-crdt-sync report is reviewed (no conflicts expected — no shared source files).