Live bounded journey — operator runbook

The "live bounded journey" lane (Lane G) drives the full CareerVector pipeline against real AI providers with real workspace state, capturing a sanitized evidence bundle. It is the only end-to-end test that proves the actual product works — every other test layer (unit, contract, mocked browser journey) uses provider doubles or curated fixtures.

It is not part of default CI. It runs only on explicit operator opt-in.

When to run it

Three legitimate triggers:

1. Before a release that touches the pipeline (extract, enrich, evaluate, tailor, render).
2. After a provider cascade change (new provider, new model, new fallback ordering).
3. After a structural change to workspace mutation paths (op catalog version bump, new sub-doc, schema migration).

Do not run it as a smoke check; the synthetic lane is faster and free.

Opt-in

The spec aborts loudly if any of the three opt-in variables are missing or incorrect:

export CV_LIVE_JOURNEY=1
export CV_LIVE_PROVIDER_OPT_IN=1
export CV_LIVE_MAX_COST_CHF=0.50   # your ceiling in CHF

Each must be set exactly. The cost cap is enforced — the spec aborts mid-run if cumulative provider spend exceeds it.

Additionally, supply at least one provider key. Groq is the default:

export GROQ_API_KEY=$(grep -oE 'gsk_[A-Za-z0-9_]+' ~/.agent/secrets/groq.md | head -1)

Any of the cascade's other providers (Cerebras, Gemini, OpenAI, …) also work if their env var is set; see lib/domain/src/providers-config.ts.

Running

Against a local dev server:

bun --filter @cv/ui dev   # in another shell
# wait for the dev server to come up
CV_LIVE_JOURNEY=1 \
CV_LIVE_PROVIDER_OPT_IN=1 \
CV_LIVE_MAX_COST_CHF=0.50 \
GROQ_API_KEY=gsk_... \
bun --filter @cv/ui playwright test e2e/journey-real/ --project=chromium --reporter=list

Against a deployed worker (Cloudflare quota policy — see wiki/content/runbooks/CI-CD.md §22):

PLAYWRIGHT_BASE_URL=https://careervector.corbet.ch \
CV_LIVE_JOURNEY=1 \
CV_LIVE_PROVIDER_OPT_IN=1 \
CV_LIVE_MAX_COST_CHF=0.50 \
GROQ_API_KEY=gsk_... \
bun --filter @cv/ui playwright test e2e/journey-real/ --project=chromium --reporter=list

Evidence output

Each run writes a sanitized bundle to:

ui/test-results/journey-real/<runId>/
  evidence.json   ← machine-readable JourneyEvidence (validated against schema)
  summary.md      ← human-readable digest

ui/e2e/helpers/live-capture.ts is the sanitizer. It removes anything matching api_key, token, secret, password, credential, authorization, cookie, set-cookie. It also refuses to write the bundle if any string in the evidence still contains a known provider key prefix (sk-, gsk_, AIza, ant-, Bearer , …). That refusal is the final safety net — if it ever fires, audit the evidence object before retrying.

Cost expectations

One full pipeline pass against Groq's llama-3.3-70b-versatile consumes roughly:

Total per run: well under CHF 0.05 for a default 4-dimension workspace on Groq. A CHF 0.50 cap leaves headroom for one full run plus diagnostic retries.

What to do if the lane fails

1. Read summary.md first — it lists which steps passed and which failed.
2. Open evidence.json for the failed step's details block.
3. If the failure is a 429 or a provider 5xx, the cascade is meant to fall over. Investigate the cascade chain in the Models tab, not the journey itself.
4. If the failure is a schema mismatch (extract returned wrong shape, evaluate returned malformed scores), capture the failing prompt + response into a new wiki/content/studies/ entry and propose a curated fixture delta for the synthetic lane.
5. Never copy a live provider response into the repo verbatim — that's captured-evidence-as-fixture which we explicitly reject (see wiki/content/studies/wiki-engine-spike.md for the broader knowledge-base reasoning).

Hard prohibitions

• This lane never runs in default CI. Verified by tools/quality/src/bucket.rs (journey-live bucket blocks --exec).
• The opt-in variables are never committed to .circleci/config.yml, .env.example, or any test fixture. They live only in the operator's shell environment.
• The evidence bundle is never committed to the repo (it's under test-results/ which is gitignored). Share it via direct file transfer or paste into the wiki manually, with another sanity scan.