CareerVector Quality Service

quality.careervector.corbet.ch is the product-facing quality service concept for CareerVector. It coordinates test buckets, quality evidence, budget policy, and synthetic workspace cleanup without depending on a specific provider, repository host, or local machine.

The service exists to answer one product question: is this change or release supported by the current testing taxonomy?

Scope

The quality service owns orchestration and reporting for quality runs. It does not own user stories, product implementation, secrets, or paid-provider credentials.

Inputs should be small and declarative:

requested bucket or gate
target environment or build identifier
optional explicit paid/live-provider opt-in
optional metadata for traceability

Outputs should be auditable:

bucket status
commands or runners used
evidence locations or artifact IDs
synthetic workspace origin and cleanup status
budget policy decisions
provider opt-in status, if any

Buckets

Bucket	Meaning	Default in service
`free-fast`	Fast zero-budget product and logic checks	Allowed
`free-slow`	Slower zero-budget integration, persistence, and convergence checks	Allowed
`browser-smoke`	Browser proof using synthetic data and mocked or fixture-backed providers	Allowed
`release-gate`	Release confidence assembled from the approved buckets	Allowed, zero-budget by default
`paid-forbidden`	Guardrail proving default runs do not call paid or quota-sensitive providers	Required for release confidence
`journey-synthetic`	Full-product browser journey using curated fixtures and provider doubles	Opt-in; not in default `run-all`. `--exec` allowed.
`journey-live`	Full-product browser journey hitting real AI providers under cost cap	Opt-in only; `--exec` is hard-blocked. Use the live-journey runner with `CV_LIVE_JOURNEY=1`.

Any paid, live-provider, or quota-sensitive run is outside the default bucket set unless the request explicitly opts in.

Importing external CI evidence

The Rust runner is not a CI replacement. CircleCI is the primary heavy runner; the quality service is the gate/ledger/policy core that records what happened.

The import subcommand brings external CI evidence into the ledger:

quality import circleci-summary path/to/workflow-summary.json [--ledger <path>]

Behavior:

Input file must be a CIWorkflowSummary JSON object (the shape emitted by scripts/ci/export-circleci-summary.ts and scripts/ci/merge-circleci-summaries.ts). Source field must be "circleci"; the importer rejects other source labels.
One ledger event is emitted per terminal job. Status mapping: success → pass, failed → fail, timed_out → timed-out, canceled → skip (with note), blocked → blocked (with note), unknown → skip (with note). In-flight statuses (running, queued) are skipped — no event emitted yet; the next import after the job finishes will record the terminal status.
Each event's run_id is a deterministic hash of workflow + commit + job + ci_status, so re-importing the same workflow produces the same run_id for unchanged jobs. Consumers can dedupe on (run_id, status).
Default bucket mapping: fast → free-fast, perf → perf, browser_* → browser-smoke, anything else → browser-smoke. (Custom mapping via --bucket-map <FILE> is a follow-up.)
The HTTP /runs endpoint cannot trigger this import. It runs only from the CLI.

Zero-budget default

Every request starts with zero budget. The service must assume that paid LLM calls, paid scraping, quota-sensitive APIs, private accounts, and mutable live-provider state are unavailable unless a request explicitly opts in.

Required behavior:

No secrets in request payloads.
No implicit paid-provider escalation.
Missing provider credentials do not break zero-budget buckets.
Paid/live-provider checks are reported as skipped or blocked unless explicitly requested.
paid-forbidden fails if a default run attempts forbidden spend.

Provider-neutral design

The service should route quality work by product capability and bucket, not by vendor name.

Provider-neutral means:

Product tests assert CareerVector behavior, not provider response quirks.
Provider adapters can have their own boundary tests, but those tests still offer a zero-budget mode.
Reports describe spend policy and capability exercised before naming any provider.
Adding or removing a provider must not change the taxonomy.

Synthetic workspace origin and cleanup

Quality runs may create synthetic workspaces. Those workspaces are product data and must be treated carefully.

Origin expectations:

Mark every test-created workspace as synthetic.
Record bucket, run identifier, suite, and created-at time.
Avoid secrets, personal paths, and user-private data in origin metadata.
Make synthetic records queryable separately from real user workspaces.

Cleanup expectations:

Cleanup is keyed by synthetic origin metadata.
Cleanup is idempotent and safe after partial failure.
Cleanup covers workspace data, queued process requests, generated documents, and browser artifacts where applicable.
Results report cleanup success, partial cleanup, or remaining artifact IDs.

Release-gate result contract

A release-gate result should answer:

Which buckets ran?
Which buckets passed, failed, skipped, or were blocked by budget policy?
Did paid-forbidden prove the default path stayed free?
Were synthetic workspaces created, and were they cleaned?
Was any paid or live-provider work explicitly opted in?
Where is the evidence needed to reproduce failures?

A release gate cannot count skipped paid work as a pass. The default release gate is useful precisely because it can run from zero-budget evidence first.