Device QA

Distributed device execution needs QA from day zero because defects here can pollute shared ad and role state. QA treats the device, session, task, lease, Observation, current ad payload cells, and global device-trust records as evidence surfaces. It does not require real Adapter code before the architecture can be tested.

This layer proves the device contract with deterministic fixtures and fake devices first. Real adapters can replace fake outputs later without changing the QA shape.

First executable slice:

jobcache/shared/src/device-contract.ts defines the v0 Zod/TypeScript contract.
jobcache/shared/src/device-fixtures.ts provides deterministic fake-device task, lease, capability, policy, and observation fixtures.
a broker/interface smoke test exercises submitObservation with those fixtures.

Scope

In scope:

Task and Observation contract fixtures.
Fake devices that emit deterministic observations.
Canary tasks that never depend on a real source.
Duplicate assignment and stale lease handling.
submitObservation schema, lease, policy, duplicate, and fresh-conflict checks.
Global device trust checks tied to Observation quality and agreement.
Resource-policy compliance for bytes, timeouts, concurrency, battery, thermal, metered network, and daily budget.
Zero-server-compute checks proving submitObservation does not execute Adapter, embedding, geocoding, LLM extraction, or graph derivation work.
QA API, MCP, UI, and ops endpoint proof that the evidence is visible to humans and agents with the same allowlist.

Out of scope:

Real adapter implementation.
Source-specific parsing logic.
Direct Cockroach or permanent object-store credentials on devices.
Workspace-private data in device tasks.
Browser profile, cookie, credential, or session reuse.

Evidence Surfaces

Durable evidence for this layer lives in the device control-plane tables from jobcache/migrations/add-device-control-plane.sql. When QA names DB-backed state, use these table names:

Table	QA use
`devices`	Durable enrolled machine identity, capability report, resource state, coarse device-trust state, heartbeat, and revocation state.
`device_sessions`	Local app/process instances, leadership state, last-seen evidence, and proof that one device does not appear as many active leaders.
`tasks`	Bounded work item, task class, payload, policy limits, freshness/deadline, attempts, and canary identity.
`leases`	Temporary device assignment, active/submitted/expired/released state, lease timing, heartbeat, and lease policy.
`ad_facts`	Current implementation table for public ad payload cells on an `ad_id`.
`ad_observation_state`	Last accepted Observation metadata per ad.
`ad_field_state`	Last observer, value hash, churn, and cooldown metadata per ad field.

Do not add extra hot submission, event, or trust-score tables for this QA layer. Submitted payloads are task output; retained raw evidence belongs in online S3-compatible object storage when it is worth keeping.

Contract Fixtures

Contract fixtures are the first executable proof. They should include valid and invalid examples for:

Capability reports.
Lease requests and lease grants.
Fetch/parse task payloads.
Embedding record payloads.
Successful observation submissions.
Partial Observation submissions.
Policy-denied Observations.
Timeout, byte-limit, and adapter-error Observations.
Quarantine candidates with mismatched hashes, missing provenance, or schema drift.

Every fixture must declare its schema_version, expected Observation, and whether submitObservation should write current cells, reject the submission, or queue verify-observation. The fixture set is the handoff contract between platform work and Adapter work.

Fake Devices

Fake devices are deterministic test actors. They have stable device_id, session_id, capability reports, resource policy, and device-trust state. They do not fetch live URLs. They return fixture-backed observations for leased tasks.

Minimum fake devices:

Device	Purpose
`fake-device-stable-a`	Happy path capability report, lease, Observation, current cell write.
`fake-device-stable-b`	Independent verifier for duplicate and canary tasks.
`fake-device-policy-denied`	Refuses tasks when policy says battery, thermal, metered network, byte budget, or daily task budget is unsafe.
`fake-device-drifted`	Emits schema drift, hash mismatch, or adapter/version mismatch to prove rejection, conflict, or trust decrease.

The server should see durable devices and sessions. It should not see local desktop windows or process identities as separate devices.

Canary Tasks

Canary tasks are reserved test tasks. They carry canary_id, use test-only source and binding identifiers, and never write production ad or role cells unless a test harness explicitly targets a disposable namespace.

Canary tasks prove:

The broker can lease a bounded task to one device.
The device submits an Observation with complete provenance.
submitObservation can write known-good current cells.
submitObservation can reject known-bad submissions.
Fresh conflicts queue verify-observation.
Ops and QA surfaces expose the same tasks, leases, current ad cells, and global device-trust evidence.

Duplicate Assignment

Duplicate assignment tests prove that two devices cannot silently produce competing current cells for the same task.

Required cases:

Same task leased twice because the first lease expired.
Same URL and content hash submitted by two devices.
Same URL with different content hashes submitted by two devices.
Same observation hash submitted twice by the same device and by different devices.
Late observation submitted after a lease has expired or been superseded.

Expected behavior:

Duplicate task enqueue returns the existing task row without a no-op write.
Idempotent duplicate Observations do not create duplicate current cell writes.
Fresh disagreements queue verify-observation.
Late Observations fail lease validation or leave newer current cells unchanged.
Expired active leases are reclaimed in a bounded oldest-first batch before ready task selection.

Invalid Submissions And Corrections

QA must prove that bad task output does not pollute current cells.

Checks:

Invalid schema rejects the submission with a reason.
Hash mismatch rejects or conflicts the submitted field and lowers global device trust as appropriate.
Policy violation rejects the submission before shared state changes.
Corrections happen through later valid Observations that update current ad payload cells with provenance.
Invalid submissions never become current ad cells or hydrate private workspace projections.

Resource Policy

Resource policy is part of the Observation contract. QA must fail when a device or server path ignores policy.

Checks:

A device that reports low battery, unsafe thermal state, metered network, or exhausted daily budget does not receive tasks that violate policy.
max_bytes, timeout_ms, and task-class concurrency caps are enforced.
Observations include the effective policy snapshot used by the device.
submitObservation rejects Observations that exceed bytes, timeout, or class policy unless the task explicitly allowed that exception.
Default concurrency remains one fetch/parse task and one embedding task per device unless policy says otherwise.

Zero-Server-Compute Guardrail

submitObservation is allowed to validate, compare, hash, record evidence, update global device trust, write current cells, and request verification. It must not execute the expensive work assigned to devices.

The guardrail fails when submitObservation or any server-side QA path performs:

Live URL fetching for a device task.
Adapter execution.
Embedding generation.
Geocoding or route lookup.
LLM extraction or summarization.
Graph derivation beyond cheap provenance and consistency checks.

This is both a cost rule and an architecture rule. Server compute may plan, lease, validate submissions, write current cells, and observe. Devices execute bounded work.

First Gates And Jobs

These gates define the first proof set. They can be implemented before real adapters exist. Do not add a QA API allowlist entry until the matching CircleCI job or pinned spec dispatch exists.

Gate	Planned job	Evidence	Pass condition
`jobcache:device-contract-fixtures`	`jobcache_device_contract_vitest`	Fixture set, schema parser output, negative-case failures	Every task, capability, lease, and Observation fixture validates or fails for the documented reason.
`jobcache:submit-observation-smoke`	`jobcache_submit_observation_smoke_vitest`	Fake-device Observation, current ad payload cell write, rejected bad submission, queued verification task	Known-good canary writes cells; known-bad canary is rejected; no expensive server compute occurs.
`jobcache:device-trust-smoke`	`jobcache_device_trust_vitest`	Global device trust updates on `devices` for stable, verifier, policy-denied, and drifted fake devices	Agreement raises or preserves trust; schema drift, hash mismatch, and policy violation lower trust.
`jobcache:duplicate-assignment`	`jobcache_duplicate_assignment_vitest`	Duplicate enqueue readback, `leases` history, duplicate Observations, current ad payload cells, and queued verification tasks	Task duplicates read back without no-op writes; idempotent Observations avoid payload rewrites; fresh disagreements overwrite current cells and request bounded verification.
`jobcache:resource-policy-smoke`	`jobcache_resource_policy_vitest`	`devices` capability/resource state, `tasks` policy limits, `leases` policy, Observation policy snapshot	Unsafe battery, thermal, metered-network, byte-budget, timeout, and concurrency cases are blocked or rejected as documented.
`jobcache:ops-device-endpoint-smoke`	`jobcache_ops_device_endpoint_smoke`	Ops API response, QA queue link, MCP-visible `devices`, `device_sessions`, `tasks`, `leases`, `brokerPath`, current ad payload cells, and global trust evidence	Capability, lease, Observation, current cell, verification, and trust state are visible without exposing workspace-private data.
`jobcache:zero-server-compute`	`jobcache_zero_server_compute_vitest`	Instrumented `submitObservation` path, blocked network/model/geocoding calls	`submitObservation` never fetches, runs adapters, embeds, geocodes, calls an LLM, or performs graph derivation for device tasks.

The first dispatchable slice should be the contract fixture gate plus submitObservation smoke. The next slice should add trust, duplicate assignment, and resource policy. Ops endpoint smoke should land as soon as a device-state projection exists.

QA Surface Contract

The QA API owns durable queue and evidence records. The MCP exposes the same discovery and trigger operations to agents. The UI shows the same queue, dispatch, gate, and evidence state to humans.

For device QA, every queue item should link to:

The gate and planned job name.
The fixture or canary tasks.task_id.
Device and session ids from devices and device_sessions.
leases.lease_id and lease timing.
submitted observation_hash and evidence references, when retained.
current ad payload cells changed by the submission.
queued verify-observation task, when present.
global device trust delta, when present.
Resource-policy snapshot.
Zero-server-compute proof.

The ops /devices response should also expose the same evidence as a cheap brokerPath summary: capability, lease, Observation, and trust counts plus explicit gap codes for the first missing step.

Human and agent triggers must use the same allowlist. The MCP must not expose a wider device QA trigger surface than the UI.

Release Rule

The day-zero gates above are not optional polish. A real Adapter family can contribute shared ad cells only after:

The contract fixture gate passes for the adapter's task and Observation shapes.
Fake-device submitObservation smoke still passes.
Canary tasks pass without server-side execution.
Duplicate assignment and invalid-submission behavior are visible in QA.
Resource-policy enforcement is proved for the task class.
Ops and QA surfaces expose the evidence needed to debug a bad observation.

If one of those checks is missing, the adapter may run in a local harness, but it must not contribute shared ad or role cells.