Device Contract

This document defines the boundary for distributed jobcache devices. It does not define adapter implementation. Source-specific fetch and parse code can be built by another agent or team as long as it obeys this contract.

Technology Decision

  • Desktop shell: Tauri.
  • Product UI: shared CareerVector Svelte/TypeScript app.
  • Native device runtime: Rust.
  • Message boundary: JSON-compatible task/observation objects, versioned by schema id.
  • Hot store: CockroachDB, accessed only by server-side DB adapters.
  • Cold evidence store: online S3-compatible storage, referenced from Cockroach by content hash and key. B2 is the current default.

Rust should own the device runtime because it is the stable place for resource checks, local queues, hashing, compression, network execution, local session coordination, and model-pack execution. TypeScript can still own UI, orchestration, task display, and pure parsing code where that is useful.

Terms

  • Adapter: code for a platform/source. It executes bounded work behind the Adapter dock.
  • Device: runtime/place where code runs. It reports capabilities and can receive leases.
  • Scraper: Adapter plus Device while executing a scrape task.
  • session: one running app instance/process on a device.
  • device leader: the local session currently allowed to advertise capacity and execute leased tasks for the device.
  • task: one bounded unit of work.
  • task class: resource/policy class for work. Current classes are scrape, chunk, enrich, embed, and verify.
  • task kind: concrete kind of work. Current kinds are discover-urls, fetch-page, parse-page, chunk-text, enrich-observation, embed-text, and verify-observation.
  • lease: temporary assignment of a task to a device.
  • Observation: task output submitted through submitObservation.
  • submitObservation: server path that validates schema, lease, and policy, writes changed ad payload cells, updates lightweight ad/ad-field observation state, updates global device trust, and queues verify-observation tasks on fresh conflict.

A chunker, enricher, or embedder is task processor + Device while executing that task class. In code and docs, prefer the precise terms above.

Non-Goals

  • No adapter code in this contract.
  • No direct Cockroach access from devices.
  • No permanent object-store write credentials on devices.
  • No workspace-private data in device tasks.
  • No browser profile, credential, cookie, or session reuse.

Device Leadership

Multiple desktop app instances may run on one machine. They must converge on one logical device identity and one active device leader.

Rules:

  • One durable device_id per enrolled machine/workspace binding.
  • Many session_id values may exist locally.
  • Only the device leader advertises capacity to the broker.
  • Only the device leader executes leases.
  • Default concurrency is one active task per task class per device unless the resource policy explicitly allows more.
  • If the leader exits, another local session may take leadership and resume advertising the same device.

The leadership mechanism is local to the device: lock file, local socket, SQLite, Tauri single-instance plugin, or another OS-appropriate primitive. The server should see one device, not many competing desktop windows.

Capability Report

The device tells the broker what it can safely do. The broker decides what to lease.

Typical capability fields:

  • device_id
  • app_version
  • device_runtime_version
  • platform
  • arch
  • battery_state
  • battery_level
  • thermal_state
  • metered_network
  • network_type
  • cpu_class
  • ram_available
  • gpu_available
  • webgpu_available
  • native_inference_available
  • supported_task_classes
  • allowed_task_classes
  • current_load
  • daily_byte_budget_remaining
  • daily_task_budget_remaining

supported_task_classes is what the device can run in principle. allowed_task_classes is what it is willing to run right now after power, thermal, network, and current-load checks. A laptop can therefore report support for enrich or embed while omitting them from allowed_task_classes on battery or under load.

Task Shape

A task is small, leased, and versioned. It gives the device enough information to do one bounded piece of work without exposing the shared ad database.

Required fields:

  • task_id
  • task_kind
  • schema_version
  • family_id
  • source_id
  • binding_id
  • url
  • url_hash
  • adapter_id
  • adapter_version
  • deadline_ms
  • max_bytes
  • timeout_ms
  • freshness_target
  • expected_observation_schema

Optional fields:

  • known_content_hashes
  • target_language
  • model_pack
  • source_hints
  • canary_id

source_id means the concrete job site or tenant. It does not mean "one adapter." A single adapter_id may serve many sources when they share a family.

Observation Shape

The device output is an Observation submission, not direct DB access. submitObservation validates the active lease and cheap invariants, then applies the latest valid field values to shared ad state.

Required fields:

  • task_id
  • ad_id
  • device_id
  • session_id
  • app_version
  • device_runtime_version
  • adapter_id
  • adapter_version
  • status
  • fetched_at
  • url
  • resolved_url
  • url_hash
  • content_hash
  • observation_hash
  • timing_ms
  • bytes_read
  • resource_policy
  • fields
  • evidence_refs
  • object_refs when cold evidence was written
  • chunk_refs when text chunks were produced
  • annotations when enrichment was produced
  • search_chunks when enriched retrieval text was produced
  • embeddings when vectors were produced during the same device task
  • errors

fields is an object keyed by shared field vocabulary. Each value contains a cell and may include evidence refs and observed_at. The Observation anchors to one ad_id; workspace-private job state never appears in the submitted Observation shape.

For embedding tasks, submit an Observation with empty fields and populated embeddings when the task only produced vectors. Each embedding is an Embedding record inside the Observation, not separate submitted output. type. Include each embedding's:

  • text_hash
  • chunk_id
  • tokenizer_version
  • model_id
  • model_version
  • quantization
  • dimensions
  • vector_hash
  • vector

Submission Rules

Devices submit Observations. Server-side submitObservation performs cheap checks, updates global device trust, writes changed payload cells, and refreshes lightweight observation state. S3-compatible cold evidence references are retained only when they are worth future replay or operator review.

submitObservation may:

  • write current payload when the submitted value differs or no current value exists;
  • leave payload unchanged on agreement while updating ad_observation_state, ad_field_state, and device trust;
  • queue verify-observation on fresh conflict;
  • lower global device trust on disagreement, schema drift, hash mismatch, or policy violation;
  • reject stale-lease, invalid-schema, or policy-denied submissions;
  • pause a source, adapter, device, or task class through broker policy.

submitObservation must not run expensive scrape, embedding, geocoding, LLM extraction, or graph derivation work itself.

Orchestration Rule

The planner creates tasks from bindings. Devices lease tasks. Adapters execute tasks on devices. submitObservation writes current cells.

source + binding
  -> planner creates task
  -> broker leases task to device
  -> device leader runs adapter
  -> device submits Observation
  -> submitObservation validates schema, lease, and policy
  -> submitObservation writes current ad payload cells
  -> submitObservation updates global device trust
  -> fresh conflicts queue verify-observation tasks

This keeps orchestration out of adapter code. An adapter should know how to execute a task; it should not decide global freshness, priority, trust, deduplication, or current cell writes.

The current broker API path is documented in broker-flow.md. It covers the implemented capability, lease, and observation endpoints; the control-plane rows; and the direct submitObservation write path into current ad payload cells and role projections.

Parallel Implementation Split

Codex owns:

  • shared Cockroach schema contract;
  • CareerVector creation-time Cockroach lookup/linking path;
  • device task/observation schemas;
  • lease/submitObservation/device-trust design;
  • resource-policy contract;
  • local session leadership contract;
  • test fixtures and contract tests;
  • docs and integration review.

Adapter implementer owns:

  • adapters;
  • family helpers;
  • fetch/parsing behavior;
  • fixture set for each adapter;
  • adapter versioning;
  • adapter conformance to the task/observation contract.

The two tracks meet through fixtures and contract tests, not through shared source-code ownership.

First Parallel Milestone

  1. Freeze task/observation schema v0.
  2. Add fake device fixtures that produce valid Observations without real scraping.
  3. Build Cockroach submitObservation path against those fixtures.
  4. Build CareerVector creation-time lookup/linking against Cockroach.
  5. Let adapters replace fake fixtures one source family at a time.

This keeps adapter work isolated while proving the shared ad boundary, DB pressure, device-trust model, and workspace integration first.

Source: jobcache/wiki/content/architecture/device-contract.md