Jobs Architecture Control

This is the control document for the jobs/ad/role cleanup. Do not perform large implementation work until the relevant invariant is represented here.

Target Model

CareerVector is moving to a sparse three-level fact tree:

role                    hidden shared aggregate
ad                      source listing for one URL/posting
job                     workspace-local tracked item exposed to users/APIs

Each ad belongs to one role. Each workspace may track many jobs. Many workspace jobs may reference the same ad.

Raw workspace storage is public workspace job data only:

workspace.jobs[job_id] = { url, attributes }

Server-only D1 tables attach a workspace job to shared data:

workspace_job_links(workspace_id, job_id, ad_id, role_id, url)

Read order for a workspace-facing attribute:

workspace job fact if present
  else ad fact if present
  else role fact if present
  else absent

Workspace APIs, UI, and MCP expose job terminology and the resolved workspace job projection. They never expose direct role/ad access or raw ad_id / role_id values.

Hidden shared ids live only in server storage: D1 link/outbox tables and shared ad/role storage. They are not part of raw workspace job entries.

Physical storage follows a private-edge / shared-HQ boundary:

Cockroach/jobcache  shared ad/role knowledge, shared work cache, vectors, shared evidence
D1/Y.Doc            workspace rung, CVL/CL, workspace-private facts, local summaries

The fact tree crosses this database boundary. This is acceptable only because the owner of each rung is explicit: Cockroach/jobcache owns shared ad/role facts; D1 owns private workspace facts. Do not make the same fact independently authoritative in both databases.

Each level is sparse. A lower value records the difference from what would be inherited. If a lower value equals its inherited value, it is redundant and can be deleted without changing the workspace projection.

The tree itself records disagreement. Do not add a disagreement table.

Identity

job_id is the public workspace job id, derived from the normalized URL in a workspace-job namespace.
ad_id is the hidden shared source-listing id, derived from normalized URL in the ad namespace.
job_id must not equal ad_id or role_id.
URL normalization and hashing are immutable once shipped.
For the first version, role_id === ad_id.
Future matching may point multiple ads at one role_id.
A workspace may track multiple jobs that later resolve to the same role. The platform must not silently merge them.

Jobcache

Jobcache is not a separate product or side cache. It is the continuous top-down ingestion/control subsystem over the same role/ad fact tree used by workspace-driven ingestion.

Two ingestion directions feed the same fact tree:

Bottom-up: a workspace imports an ad. The system first asks whether the ad is already known; on a hit, the workspace gets a reference and resolved projection without repeating expensive work.
Top-down: jobcache crawls/searches the web, creates or refreshes ads, enriches them directly at ad level, and clusters ads into roles.

Jobcache also coordinates expensive work caches: scrape/readout, extraction, embedding, geocoding, route lookup, and matching. These caches are keyed by stable inputs so the platform does not pay for the same work twice.

The target jobcache runtime is a continuous ingestion service, not a product surface built around manual scrape buttons. It should maintain source/frontier rows with cadence, backoff, leases, and failure state; adapters should feed candidate ads into the shared ad/role fact nodes; workspace imports should feed the same queue when a user brings a new URL from the floor; and the control pane should add, pause, inspect, and repair sources without changing the data model. In this model jobcache is the top-down end of CareerVector, while the workspace app is the user-facing bottom-up end.

Current guarded operator/debug controls are acceptable only while hidden, quota guarded, and treated as resource controls over the queue, not as the product contract. The durable product path is the planner/task/device/observation loop; old direct scrape controls should disappear or become narrow commands over that same queue.

Write Routing And Compression

Writers emit candidate facts or observations. Field definitions decide whether and how those facts may compress upward.

Workspace commands write workspace job facts and workspace state.
Jobcache crawler/extract/enrich writes ad-level candidate facts.
Matching and reducers may write role-level facts.
Custom columns and evaluations can exist in the fact-tree system at workspace level. They do not move upward unless their field definition has a named reducer.
CVL/CL content is a workspace-private, collaborative, fact-bearing artifact. It lives in D1/Y.Doc and does not move upward unless an explicit user consent rule or aggregate reducer says so.
Notes, files, conversations, and UI state remain workspace-private and do not move upward unless a later field definition explicitly changes that.

No value moves upward merely because a runtime flag says it is visible. Movement requires a field definition and reducer in code.

Transaction Boundaries

Atomicity requirements:

Creating a workspace job must not leave it without the required ad and role nodes in the server path.
The browser optimistic path may create local workspace.jobs[job_id], but the server path must attach the shared ad/role ensure or outbox hook.
Cross-sub-doc writes must be planned and committed as batches. Do not fake atomicity by issuing unrelated sequential ops when the user-visible invariant requires all-or-nothing.
D1 and Cockroach are not one transaction. Cross-boundary writes require an outbox/idempotency protocol. If one side commits and the other does not, the pending or failed state must be visible; no silent split-brain.

Compression invariant:

Pure compaction must not change any workspace projection.
Reparenting an ad under a different role must preserve every workspace projection unless the operation is explicitly a synthesis update. Preserve first by materializing ad facts where needed, then compact redundancies.

Current Mismatch

The core mismatch blocking cleanup:

lib/mutations, workspace-client, web routes, MCP, and dashboard code must agree that workspace ops are job.* and the workspace sub-doc is jobs.

This must be resolved before broad web edits.

Forbidden Shortcuts

Do not make two current workspace storage shapes.
Do not add fallback reads that silently choose whichever shape exists.
Do not restore job_id_allocator as the primary identity path.
Do not copy raw private workspace data into Cockroach.
Do not let CVL/CL content, notes, conversations, files, or UI state move upward without an explicit field definition plus user consent rule or reducer.
Do not let custom columns or evaluations move upward without a field definition and reducer.
Do not change URL normalization/hash rules without an explicit content migration plan.

First Implementation Slice

Proposed first slice:

Align op catalog, workspace-client, and server validation on job.* ops and the jobs sub-doc.
Update tests to fail loudly on non-job workspace tracking ops.
Do not touch dashboard/CVL/UI yet except compile-required type fixes.

This slice establishes the client/server vocabulary before UI rewiring.