QA Object POV Dashboard Design

This note is the design checkpoint before rewiring the QA dashboard.

The dashboard must not start from "what CircleCI shows" or "what table is easy to render." It starts from the graph object or QA operating loop being watched. Behaviour, Suite, and Test are real hub kinds. Queue is the execution ledger. Gates, flakes, dispatches, and runner state are facets, evidence, or actions attached to those objects.

Governing Rule

Use the sanctioned relationships as the page contract:

behaviour
  must connect to: idea
  may connect to: implementation, suite, anomaly, journal

suite
  must connect to: behaviour, at least two tests

test
  must connect to: suite, implementation

For QA screens, ignore anomaly/event except as ops-owned context. A failed test is QA evidence, not an ops event or anomaly.

Every object page should answer:

What am I?
What must I be connected to?
What am I actually connected to?
What proof or execution evidence do I carry?
What is missing, stale, noisy, weak, or suspicious?
What should a human or agent do next?

API First

The API should expose object POV resources, not UI tables:

GET /v1/behaviours/:id/proof
GET /v1/suites/:id/proof
GET /v1/tests/:id/proof
POST /v1/investigate/changes
GET /v1/graph/qa/neighborhood
GET /v1/queue
GET /v1/queue/:id/evidence
GET /v1/ci/providers
GET /v1/qa/worries?kind=behaviour|suite|test

Each proof response should include:

• the object itself
• direct sanctioned neighbours
• indirect explanatory path one hop further where useful
• compact execution evidence
• graph validity flags
• suggested next actions

For behaviour proof, the API should expose idea_doors next to the behaviour itself. A behaviour without an idea door is not a UI omission; it is a graph gap that should stay visible until the semantic index or source-backed graph can ground the behaviour in an idea.

Execution evidence is a property/record:

last_result
last_run_at
last_commit
last_duration_ms
flake_score
ci_provider
provider_pipeline_id
provider_workflow_id
artifacts

Do not call these QA records event or anomaly.

MCP Second

MCP tools should mirror the API objects and be useful without the UI:

inspect_behaviour
inspect_suite
inspect_test
investigate_changes
get_qa_neighborhood
list_queue
inspect_queue_item
get_ci_providers
list_qa_worries
run_suite
run_tests
explain_qa_path

The agent should be able to ask:

Why is this test invalid?
Which implementations does this suite prove?
Which behaviours lost proof after my code change?
Which tests are stale or flaky but still gate important behaviour?

UI Third

The tab order stays:

Behaviour | Investigate | Queue | Suites | Tests

The UI should make the graph visible without forcing users to think about graph theory. Every card or row should show:

• identity
• mandatory edges
• evidence
• worry level
• next action

Useful outside patterns:

• CI tools surface status, duration, failure, and flake data as first-class facets.
• Observability tools make comparison and drill-down cheap: select a suspicious subset, compare it with the baseline, then pivot.

Our product difference is that every pivot follows the project graph, not only CI execution history.

Behaviour POV

A behaviour is an actionable promise. It asks: "Is this promised behaviour defined, grounded in an idea, implemented somewhere, and proven by suites?" Behaviours should be built around Gherkin shape, and the UI should make that shape visible rather than treating behaviour as plain prose.

                 Idea
                  ▲
                  │ must express
            ┌────────────┐
            │ Behaviour  │
            └────────────┘
              ▲        ▲
   verifies   │        │ realizes / supports
            Suite   Implementation
              ▲
              │ contains
             Tests

The behaviour page should worry about:

• no idea edge
• no suite proving it
• suite exists but has fewer than two tests
• linked suites are stale, failing, or noisy
• implementation path missing or only implicit
• Given/When/Then shape is vague
• behaviour claims too much for its proof set

Interface idea:

BEHAVIOUR: workspace imports stay resumable
Promise: Given an interrupted import, when the user resumes, then no job is duplicated

Graph:   Idea ✓   Suites 2   Implementations 3
Proof:   latest suite pass 18m ago   flake 0.0   stale no
Worry:   one implementation has no direct test
Actions: open idea | open suite | run proof | inspect gap

The list view should sort by worry before alphabet:

invalid -> unproven -> stale -> noisy -> failing -> healthy

Suite POV

A suite is a proof set. It asks: "Do these tests together prove the promised behaviour, and is this battery runnable and trustworthy?" Suites may live in git-backed YAML, but they still need to be represented through the QA API/MCP/UI as first-class suite objects.

            Behaviour(s)
                 ▲
                 │ verifies
             ┌───────┐
             │ Suite │
             └───────┘
              ▲  ▲  ▲
              │  │  │ contains
            Test Test Test
              │  │  │
       Implementation(s)

The suite page should worry about:

• no behaviour edge
• fewer than two tests
• tests without implementation edges
• tests that all hit the same narrow path while the behaviour is broader
• too many tests for one small behaviour
• stale latest execution
• slow or flaky members
• CircleCI binding missing when the suite is meant to be runnable

Interface idea:

SUITE: dashboard modal proof
Verifies: modal save/cancel/validation behaviour

Coherence: 4 tests, 3 implementations, 1 behaviour
Execution: CircleCI browser_dashboard, pass, 11m ago, 42s

Tests:
  ✓ save persists state        checks dashboard-modal-workflow
  ✓ cancel restores draft      checks dashboard-modal-workflow
  ⚠ validation blocks save     checks dashboard-validation-workflow
  ✓ escape closes modal        checks keyboard-modal-workflow

Actions: run suite | open behaviour | inspect weak test | split suite

Suites should render as a matrix more than a plain table: behaviours across the top, tests down the side, implementations as chips in the cells. Missing mandatory edges become empty red cells.

Current rev1 checkpoint:

GET /v1/suites/:id/proof
inspect_suite
/suites/:suiteId

The rev1 suite proof endpoint returns the suite, behaviour doors, test witnesses, counts, and low-confidence implementation doors derived from the test catalog. This is deliberately a QA projection over the universal graph; Journal and Ops projections are out of scope for this sprint.

Test POV

A test is a witness. It asks: "What implementation do I check, which suite needs me, and what did I last prove?"

                    Behaviour
                       ▲
                       │ verified by
                    Suite(s)
                       ▲
                       │ contains
        checks     ┌────────┐     evidence
Implementation ◀── │  Test  │ ───────────────▶ CircleCI result
                   └────────┘
                       │
                       ▼
                 file / symbol / owner

The test page should worry about:

• no suite edge
• no implementation edge
• not run on current commit
• stale last run
• flaky, slow, or frequently skipped
• failing while related tests pass
• passing while its suite still fails
• overly broad or overly narrow title compared with implementation edge
• no clear file/symbol anchor

Interface idea:

TEST: modal save blocks invalid salary
Status: pass, fresh, not flaky, graph-valid

Checks:       dashboard-modal-validation
Belongs to:   dashboard-modal-proof, dashboard-e2e-suite
Explains:     salary edit behaviour -> dashboard idea
Evidence:     CircleCI #4821, 3.2s, main@abc123, trace + screenshot
Worries:      none
Actions:      run | open code | open suite | open behaviour

The test list should become an inspection board, not only a catalog:

Invalid first: missing suite or implementation
Then: stale, flaky, failing, slow, suspicious
Finally: healthy witnesses

Current rev1 checkpoint:

GET /v1/tests/:id/proof
inspect_test
/tests/:testId

The rev1 test proof endpoint returns the catalog test, its suite membership, the behaviours reached through those suites, implementation link status, and recent execution records. Test titles in the catalog, suite proof rows, and investigation results now navigate to the test object rather than only filtering the catalog.

Queue POV

Queue is not a fourth QA object and not product Ops. It is the QA execution ledger and operational work queue for limited verification capacity.

It asks: "What did the verifier enqueue, what is running now, and what will it do next?"

Queue should present a vertical time axis:

future
  queued suite/test execution
  queued suite/test execution

now
  running workflow/job
  running workflow/job

past
  completed workflow/job
  completed workflow/job

The page should worry about:

• long queues that humans did not expect
• AI agents ordering hours of testing without visibility
• queued work with low value blocking high-value checks
• running work stuck or canceled
• repeated failed/canceled execution attempts
• congestion against limited QA capacity

The primary UI should be an execution list with a queue facet, not a graph object page. The graph still supplies doors:

queued dispatch -> suite/test
running queue item -> queue detail and provider evidence
completed failed queue item -> failed tests -> suites -> behaviours

Runner readiness/capacity belongs in the top-right RunnerLight, not in the Queue page body. Queue shows the execution timeline. RunnerLight shows whether the machine can take work.

API/MCP implications:

GET  /v1/queue
GET  /v1/queue/:id
GET  /v1/queue/:id/evidence
POST /v1/queue/dispatch
GET  /v1/ci/providers

list_queue
inspect_queue_item
dispatch_job
get_ci_providers
reprioritize_qa_queue_item
cancel_qa_queue_item
explain_qa_congestion

Investigate POV

Investigate is not a graph kind. It is a traversal workbench:

changed artifact -> tests -> suites -> behaviours -> implementation door

The mature Investigate view should query the graph first and use path heuristics only as a fallback.

Current rev1 checkpoint:

POST /v1/investigate/changes
investigate_changes
/investigate

Rev1 accepts changed paths or a diff, applies the existing path/workspace heuristic, and returns full test proof paths instead of only test ids. The UI renders each matched test as doors into Test, Suite, Behaviour, and Wiki Atlas implementation views. It is still QA-only; Journal and Ops projections remain out of scope for this sprint.

Naming Guardrails

• Use quality record, execution record, or gate record in QA.
• Reserve event and anomaly for ops.
• Do not model a CI job as the suite itself. A CircleCI job can execute a suite today, but the suite must remain stable if another CI provider executes it later.
• Direct behaviour-to-test links are a shortcut to remove. Behaviour proof is mediated by suites.
• Keep the shared top-right connection ribbon across Wiki, Ops, and QA. It should show git/db/graph connection state. QA additionally shows RunnerLight because runner readiness is specific to QA execution capacity.

External Inspiration Checked

• Datadog CI Visibility emphasizes status, duration, failures, traces, and drill-down from pipeline/job/test results.
• Buildkite Test Engine exposes flaky tests as first-class test analytics.
• Honeycomb BubbleUp is a useful mental model for suspicious subset vs baseline comparison, but our drill-down follows graph relationships.