gilde

← All craftbooks

Test Coverage Review

Review and QAevalv1.1.0released 2026-07-19workflow: build-loop

Review a codebase or module's test suite for real coverage and quality, then produce a gap report and a prioritized backlog of tests to add. First scopes the units under review and locks a coverage standard distinguishing line coverage from behavioral coverage (critical paths, branches, edge cases, error handling, and regression protection), then audits the existing tests for what they actually assert and which behaviors and branches are untested or weakly tested, then writes a report ranking the highest-value missing or low-quality tests with a concrete description of each. Use this for a test coverage review, finding test gaps, auditing test quality, deciding what to test next, or assessing whether a module is well-tested — it distinguishes meaningful behavioral coverage from a green coverage number and returns a prioritized testing backlog rather than 'add more tests'.

Steps

Entry step: scope. Each step names the specialist role it wants; the full working prompt is expandable.

  1. Scope the coverage standardreviewerentry

    lock units in scope + a behavioral coverage bar

    Show working prompt
    Step 1: Call `map_repo`, `list_file_issues`, and targeted `search_code` queries to identify the modules under review, their public behavior, critical paths, risky files, and the tests that claim to cover them. Record indexed review coverage so unreviewed code is visible. Step 2: Run the existing test and coverage package scripts if safely available; capture commands, exit codes, and line/branch/function numbers as starting signals, not the goal. Step 3: Write a coverage checklist distinguishing LINE coverage from BEHAVIORAL coverage: every critical path has an outcome assertion; each important branch is exercised; boundary/error/cancellation/concurrency cases are tested where relevant; tests assert behavior rather than mere execution; and a regression in core logic would fail a test. Step 4: Set a risk-based target for THIS code and write the behavioral surface, measured evidence, index/tool limitations, and bar to `write_task_note` AND `notes/scope.md`. No audit yet.
  2. Audit the test suitereviewer

    find untested behavior + weak assertions

    Show working prompt
    Step 1: Map the existing tests to the behavioral surface — for each unit, what is tested and what does each test actually ASSERT. Step 2: Flag the gaps: untested public functions, untested branches, missing edge cases (empty/null/boundary/error), and missing regression protection on core logic. Step 3: Flag WEAK tests — tests that only check 'it runs' without asserting outcomes, over-mocked tests that would pass even if the real logic broke, snapshot-only tests of meaningful logic, and tests coupled to implementation rather than behavior. Step 4: Cross-check the coverage numbers: where lines are 'covered' but no assertion verifies the result, call it out (covered ≠ tested). Step 5: For each gap, draft the test to add: the unit, the scenario/input, and the expected assertion. Step 6: Rank by value — core/critical logic and error paths first. Good looks like: 'parseAmount() has no test for negative or non-numeric input → add a test asserting it throws/returns Err', not 'add more tests'. Stage the gap list to `write_task_note`. Do not write the report yet.
  3. Write the coverage reportreviewer

    prioritized test backlog + quality findings

    Show working prompt
    Step 1: Open `coverage-review.md` and write a '## Summary': the coverage numbers, an honest behavioral-coverage assessment, and the biggest gap. Step 2: Add a '## Missing tests' backlog ranked by value; each entry: the unit, the untested scenario, and the assertion the new test should make. Step 3: Add a '## Weak tests' section listing low-value/misleading tests with how to strengthen them. Step 4: Add a '## Well-covered' section noting what is solidly tested (so effort goes where it's needed). Step 5: Add a '## Quick wins' callout for high-value, easy tests to add first. Every recommendation must name a concrete test and its assertion — no 'increase coverage'. On a loop-back, fix only the named gaps. `write_task_note` the report path and the count of missing high-value tests.
  4. Evaluatereviewer

    Grade the deliverable against every acceptance criterion. All pass → finish; any fail → loop back and fix the gap.

    Show working prompt
    Open `coverage-review.md` and grade it against the scope coverage bar. Check EACH criterion and write PASS/FAIL with a one-line reason: (1) the review distinguishes line coverage from behavioral coverage rather than chasing a number; (2) untested critical paths, branches, edge cases, and error paths are identified; (3) each missing test is described concretely with the scenario and the assertion to make; (4) weak/misleading tests are flagged with how to strengthen them; (5) the backlog is prioritized by value with quick-wins called out; (6) a well-covered section keeps effort focused; (7) no vague 'add more tests' filler. If any criterion fails, name the exact gap and loop back.

Then route — this is the whole point of the loop:

- **Every criterion PASSES →** call `advance_task_step({ ref, stepId: "evaluate", next: "finish" })`.
- **Any criterion FAILS →** write the specific gaps to notes, then call `advance_task_step({ ref, stepId: "evaluate", next: "report" })` to loop back. The builder fixes exactly those gaps.

Never route to `finish` while any criterion is unmet. The build phase's completion gate already blocked a grossly-incomplete deliverable; your job is the judgment an automated check cannot make (does it actually work, read well, look right). After ~3 unproductive loops, stop and report DONE_WITH_CONCERNS so the user can step in.
  5. Finishdeveloper

    All acceptance criteria met. Stamp a short summary and report DONE.

    Show working prompt
    Every acceptance criterion passed. Write a one-paragraph DONE summary to task notes via `write_task_note`: what was built, the deliverable path(s), and a one-line confirmation that each criterion is met. Then report DONE.

Triggers

Phrases that suggest this craftbook to a crew.

Toolsets

Source

View this craftbook on GitHub · MIT license