JobCache Ops — Concrete Views

JobCache is CareerVector's sister product, sharing the role → ad → workspace job fact tree (CLAUDE.md sibling repo notes; jobcache repo .agent/AGENT.md). Today it focuses solely on extracting job ads. Its Ops surface therefore reads as a business cockpit for corpus ingestion, not a user-friction cockpit.

Current shell contract: JobCache Ops uses the same primary tabs as CareerVector Ops:

Anomalies | Events | Infra | Inspector | Sysadmin

The source matrix, ingestion tape, device fleet, corpus panels, and recovery actions described below are JobCache-specific content inside those canonical tabs. They are not a separate one-page product shell.

The audience is different from CV's:

Status (status.jobcache.corbet.ch) speaks to all three audiences in aggregate. Ops (ops.jobcache.corbet.ch) is the operator's working surface and the business stakeholder's gated read-only window.

What is different from CV

Some CV cockpit primitives apply directly: infra checks, queue mix, quota cost, event tape. Reuse them. Diverge where the product shape differs:

JobCache does not have "workspaces" in the CV sense. It has sources, runs, postings, roles, and matches. The cockpit reflects that.

Canonical tab placement

Anomalies
  source silence
  schema drift
  trust conflict loops
  quota danger
  device abuse

Events
  ingestion tape
  scrape/run events
  observation events
  lease/device events
  operator journals

Infra
  API health
  CockroachDB
  online S3/B2 snapshots
  device fleet
  provider quotas
  cost per ad
  freshness and corpus pulse

Inspector
  source dossier
  ad/role dossier
  device dossier
  recent runs and samples
  linked QA/Ops/Wiki context

Sysadmin
  pause/resume source
  rerun failed scrape
  reset drift
  pin note
  quarantine device/ad

The older one-page sketch with header panels, source matrix, ingestion tape, and audit log is retained below as a panel inventory. Implementation should mount those panels inside the canonical tabs and keep the shared Ops header/chrome.

Infra panels

Infra check rollup

Source: JobCache Ops API health and runtime/device observations. There is no canonical central heavy worker in the target architecture; scraping and embedding work run on leased devices or explicitly configured workers of last resort. The Ops surface should summarize that fleet rather than assume one server worker.

Each row: name, state, latency, last-checked timestamp.

Status projection: API, CockroachDB, online S3/B2, and aggregate device fleet state only. No per-provider quota, per-source volume, or latency detail.

Corpus panel

Source: JobCache API/read-model counters and daily snapshots. Runtime counters from devices may back this, but the UI should read normalized API shapes rather than Prometheus text or a specific worker.

Status projection: all of these. "N postings, N today, dedupe X%" is the public headline ("How many companies are hiring through us?").

Freshness panel

Source: query CockroachDB for MAX(scraped_at) and PERCENTILE_CONT(0.95)(now() - scraped_at) over visible ads.

The freshness panel is the corpus's pulse. A stale corpus is a dying business.

Status projection: median + a one-word adjective ("fresh", "aging", "stale"). p95 and stale share are operator-internal.

Cost / ad panel

Source: cost ledger queries (we keep counters per provider call; need to add a per-ad cost rollup if not present).

Cost per ad inserted is the unit economic. If it crosses a threshold the operator should know immediately.

Status projection: none. Cost is operator-and-business-only.

Source matrix

The cockpit's centerpiece. One row per known source, sortable by freshness or activity.

Columns:

Source operational state

The state rules are aggressive on detection: a source going from producing 300 ads/day to 0 is more alarming than a stuck workspace. For JobCache, silence is failure.

Filters

• State — show only red/yellow sources during incidents.
• Scraper class — direct, browserbase, firecrawl, etc.
• Source category — direct-jobs-board, aggregator, ATS.

Source detail drill-down

Click a source → side panel.

• Recent runs: time, duration, postings inserted/deduped/skipped, errors, scraper class used.
• Schema-drift detail: which extractor field failed, on which postings, what the LLM returned (since this is internal scraping, no PII concerns), what the canonical schema expects.
• Cost breakdown for this source over the last 24h.
• Output examples: a sample of postings inserted in the last hour (deep-link to jobcache web view).
• Recovery actions: re-run last failed scrape, pause this source, reset schema-drift detection, pin operator note.

Every recovery action writes a row to a new operator_actions audit log (operator id from session, action, target, timestamp, optional note). The audit log is admin-only and rendered as the "Audit log" cross-cut tab in the cockpit.

Audit log schema

Table: operator_actions (CockroachDB, append-only). See the migration at jobcache/migrations/add-operator-actions.sql for the canonical SQL — column list, types, and indexes are intentionally not duplicated here so the migration file remains the single source of truth.

The schema mirrors CV's operator_actions (careervector-cockpit.md → Audit log schema) with two JobCache-specific differences:

1. CockroachDB-flavored SQL: STRING instead of TEXT, INT8 for ts_ms, JSONB for result_details (CRDB has native JSONB; D1 does not).
2. target_type enum reflects JobCache's primary entities: 'source', 'scrape_run', 'posting', 'system' (vs CV's 'workspace', 'process_request', 'system').

Canonical action enum. The JobCache cockpit emits exactly these action values (extend by adding rows here and updating the writer in the JobCache api.ops worker; the column itself is unconstrained STRING so the set can grow without a schema bump):

Indexes cover the three common operator queries: recency (idx_operator_actions_ts), per-target history (idx_operator_actions_target), and per-action filter (idx_operator_actions_action).

Apply the migration (against the JobCache CRDB cluster — see secrets/cockroach.md for the connection URL):

cockroach sql --url "$JOBCACHE_DATABASE_URL" --file jobcache/migrations/add-operator-actions.sql

Or paste-and-execute via the CockroachDB Cloud console SQL shell.

Retention. Append-only by design. There is no UPDATE or DELETE path from the cockpit, and no automatic purge job. If log volume becomes a concern, add a separate retention job — out of scope for this table.

Ingestion tape — live

Source: same SSE Fanout pattern as CV (architecture B in architectures.md), pointed at JobCache's queue tables. The schema differs:

Same operator pulse as the CV tape, different vocabulary.

Status projection: an aggregate-only public version. "Postings ingested per hour" sparkline; no source attribution.

Cross-cuts (top-bar "more")

• Role-clustering quality — confidence distribution of recent ad → role assignments. Manual override rate (when corpus consumers reject a clustering).
• Embedding cost — token spend on embedding generation per day, by model.
• Workspace consumption — which CV workspaces pulled which ads in the last 24h. Anonymized by default (workspace-id hashed); unhashed only if the workspace is yellow/red tier. Pull-through rate: ads that became workspace jobs / ads pulled.
• Source coverage gap — known role families × known geographies, with a heat map of how many ads we have. The negative space is where the corpus is weak.
• Audit log — operator_actions for JobCache.

What status (public) gets

status.jobcache.corbet.ch continues at 60s ISR. The page is already further along than CV's because readJobcacheHealth already returns rich Health snapshots; status just renders them.

What status drops or never adds:

• Per-source volume (contractual concern; some sources do not want their volume public).
• Cost data.
• Workspace consumption patterns.

Architecture notes specific to JobCache

• CockroachDB is the canonical store for shared ad/role state, unlike CV where D1 is canonical. Most cockpit queries hit Cockroach, not D1. The architecture options in architectures.md apply; substitute CockroachDB queries where D1 is mentioned. CRDB query latency is higher than D1, so caching the snapshot more aggressively at the edge (60s on status, 30s on ops) is fine.
• Normalized Ops API shapes are the product boundary. Prometheus-style counters may still be a useful backing source for runtime/device metrics, but UI and MCP read normalized API payloads. The cockpit should not depend on Prometheus text, a specific worker, or direct vendor responses.
• Device/runtime fleet replaces the central heavy-worker assumption. Scraping, chunking, and embedding work run on leased devices or explicit workers of last resort. Ops probes the fleet, leases, and capacity rather than one named background worker.
• Browserbase / Firecrawl quotas are external rate ceilings that matter as much as CF quota. Surface them in the infra panel.

What this leaves to follow-up work

• Per-source freshness SLA — once we have business stakeholders, some sources will have contractual freshness commitments. The cockpit will need per-source SLA tracking with breach alerts.
• Source-level cost vs. value — when commercial pricing exists, cost per ad becomes value per ad (ads consumed × consumer pricing). The cost panel evolves into a margin panel.
• Embedding/match diagnostics — quality of matches in workspace pulls (precision/recall vs. operator-labeled ground truth). Out of scope for the first cut; will become important once matching is a tunable business surface.
• Public corpus browser on status — a paginated, anonymized, rate-limited view of recent insertions. Useful for partners. Not in v1.