Audit: {provider, model} vs {providerId, model} Schema Fitness

Date: 2026-05-11 Branch: cvl-node-tree-direct Scope: All TypeScript files in the monorepo (excluding node_modules, .d.ts)

Background

Two distinct object shapes coexist in this codebase:

The resolveChain() function converts ChainSlot → ProviderSlot by reading config.providerKeys[slot.provider] and renaming the field:

result.push({ providerId: slot.provider, apiKey: key, model: slot.model });

So providerId is correct on resolved/runtime objects; provider is correct on stored/seed objects. Confusion arises where test fixtures construct objects that are meant to be stored ChainSlots but mistakenly use providerId.

Findings Table

Inverse Mistake Check

Checking for places that use {provider, model} where the consumer expects {providerId, model}:

• lib/domain/tests/bundle-equivalence.test.ts: chainConfig.chains entries correctly use {provider, model} (ChainSlot); chainResolver stubs correctly return {providerId, apiKey, model} (ProviderSlot). No confusion.
• lib/domain/tests/pipeline-stages.test.ts: makeChainResolver correctly converts s.provider → { providerId: s.provider, apiKey: ..., model: s.model }. No confusion.
• No inverse mistake instances found.

Wrong-Shape Instances — Proposed Fixes

Instance 1: mcp/test/tools.test.ts:278-289

What: Test fixture seeds chain_overrides for a job with { providerId: 'mistral', model: 'mistral-small-latest' } inside evaluate: [...]. Since chain_overrides is Record<capability, ChainSlot[]> per lib/domain/src/types.ts:60, the slot shape must be { provider: string, model: string }.

Why the test passes today: list_jobs in the MCP tool round-trips chain_overrides by parsing it from JSON and returning it to the caller as a plain object without schema-validating the slot contents. The expect(jobs[0].chain_overrides).toEqual({ evaluate: [{ providerId: ... }] }) assertion then matches what was seeded. The test never exercises the slot through flattenCascade or resolveChain, so the wrong field name never causes a failure — but the fixture is still wrong and would produce incorrect runtime behaviour (the slot would be filtered out by pruneOrphanedChainSlots because slot.provider would be undefined).

Proposed fix:

- chain_overrides: JSON.stringify({ evaluate: [{ providerId: 'mistral', model: 'mistral-small-latest' }] }),
+ chain_overrides: JSON.stringify({ evaluate: [{ provider: 'mistral', model: 'mistral-small-latest' }] }),

- expect(jobs[0].chain_overrides).toEqual({
-     evaluate: [{ providerId: 'mistral', model: 'mistral-small-latest' }]
- });
+ expect(jobs[0].chain_overrides).toEqual({
+     evaluate: [{ provider: 'mistral', model: 'mistral-small-latest' }]
+ });

Summary Verdict

How many tests are silently broken?

One test in mcp/test/tools.test.ts (the list_jobs parses JSON text blobs test at line 272) seeds a wrong-shape fixture and asserts the round-tripped wrong shape. It passes because list_jobs never schema-validates ChainSlot contents. If the tool were updated to validate or forward the chain_overrides through flattenCascade, the slot would be silently dropped (pruned as orphaned — no slot.provider key).

The previously-reported bug in fallback/server.test.ts is already fixed: the chain_overrides fixture at line 992 now correctly uses { provider: 'mistral', model: 'mistral-small-latest' } (with an explanatory comment at line 989).

Recommended next action

1. Fix mcp/test/tools.test.ts lines 278 and 289 (one-line patch each, shown above).
2. No production code changes needed.
3. Consider adding a ChainSlotSchema.parse() or z.array(ChainSlotSchema) validation at the list_jobs deserialization boundary in the MCP tool to catch future shape drift at the ingestion point.