Pull Request Review

Review and QAevalv1.9.2released 2026-09-05workflow: review

Staff-engineer-style PR review over a complete local PR corpus. Defaults to the open pull request for the project's checked-out branch when one exists, accepts an explicit PR number from any branch, reviews changed files in bounded batches with mechanical coverage accounting, verifies cross-file claims against the checkout, and produces a structured local verdict. Requires a GitHub-connected project.

Steps

Entry step: scope. Each step names the specialist role it wants; the full working prompt is expandable.

Map the pull request corpusreviewerentry

Read the launch-time PR overview and manifest, capture CI/check evidence, and publish the deterministic review batches the runtime fans out over.

Show working prompt

**You are reviewing PR #{{number}}, not authoring it. Never modify project source in this craftbook.** The runtime already mirrored the selected PR into `{{corpusScope}}`; this corpus, not a single live diff call, is the authoritative review surface.

**The fanout input is already published.** On entering this step the runtime published the manifest's complete batch array to `{{workPath}}/pr-review/batches.json` and enriched each batch with its exact current `records` paths. That file is done — do not write it, and do not "fix" it. Retyping 500+ exact paths through a tool call is what this step used to ask for, and it cannot survive a per-turn output cap.

Your job is the evidence and the scope note.

1. Call `list_artifacts({ path: "{{corpusScope}}", recursive: false })` and read the overview Markdown record it returns. Read the manifest directly at `{{corpusScope}}/attachments/001/pr-{{number}}-files.json`. Do not recursively list the corpus.
2. Call `read_artifact({ path: "{{workPath}}/pr-review/batches.json" })` and note the batch count and ranges from what is actually on disk.
3. Call `github_check_status` for the overview's head ref and record whether checks are success, failure, pending, or unknown. This is evidence, not a substitute for reviewing the diff.

Then write ONE task note beginning `## Scope — PR #{{number}}`. Include title, URL, head → base, draft state, exact changed-file count, manifest path, batch count/ranges, CI/check status, existing-review themes, and 3–6 review themes weighted by focus `{{focus}}` and intensity `{{intensity}}`. Derive every path and count from the manifest and the published batches — never from memory. End by calling `advance_task_step`.

These deliverables live in the project's artifacts drawer — read them with `read_artifact`; the shipped workspace stays untouched.

Fan the batches out to the review crewreviewer
Spawn one child reviewer per published batch. The runtime performs the fanout with no model turn; the crew is the work.
Show working prompt
```
The runtime spawns one child reviewer per entry in `{{workPath}}/pr-review/batches.json`, each in its own session holding only that batch's records. No turn is needed here.
```

Merge the batch ledgersreviewer

Wait for the crew while the runtime deterministically rebuilds the run-wide ledger from exact per-batch coverage shards, then prove both corpus completeness and shard provenance.

Show working prompt

The runtime has rebuilt `{{workPath}}/pr-review-coverage.json` from every valid `coverage-N.json` shard currently on disk. Do not write or edit that ledger yourself — it is deliberately runtime-owned so a large pull request never has to pass hundreds of exact paths through one model tool call.

Call `advance_task_step` now. The gate will compare the ledger against the complete connector corpus and independently prove that it is exactly the union of one valid shard per published batch. If reviewers are still working, it will reject and name the missing batches; on the fresh activation the runtime will rebuild the ledger again. Do not review files yourself or add paths to close the gap. A batch that remains stuck belongs in a task note, not a fabricated coverage claim.

The ledger and shards live in the project's artifacts drawer; the shipped workspace stays untouched.

Synthesize the reviewreviewer

Synthesize every batch's observations into a cited report, re-verifying cross-file claims and keeping CI status distinct from code-review judgment.

Show working prompt

The coverage gate has proved that every changed path in PR #{{number}} was reviewed. Read the Scope note and the artifact `{{workPath}}/pr-review-coverage.json`, then call `list_artifacts({ path: "{{workPath}}/pr-review", recursive: true })` and read EVERY `observations-N.md` shard with `read_artifact`. There is one per batch and each holds findings the others do not — synthesizing from a subset silently drops whole batches of review.

Re-check every critical or major candidate against the relevant workspace source with `find_symbol`, `search_code`, `grep_files`, `read_file`, or `read_files`; call `github_check_status` again if the earlier status was pending. Drop any candidate that is not supported. Reconcile duplicates: separate reviewers can raise the same defect from two batches — merge those into one row. Do not repeat an existing PR comment unless it still needs action and you explicitly say it was already raised.

Cite every finding as `path:line` using a changed path and a new-side diff line. Any critical or major finding requires `request-changes`; otherwise use `approve`. CI success is evidence that the checked revision compiled/tested as configured, but it does not erase logic findings. CI unknown/pending is not itself a code defect.

Write `{{workPath}}/pr-review.md` in ONE `write_artifact` call using exactly this skeleton:

```
# Pull Request Review — PR #{{number}}: <title>

## Summary
<2–6 sentences: what changes, overall risk, existing-comment coverage, and CI/check status. Say "No findings." when there are none.>

Coverage: <reviewed count>/<changed-file count> changed files across <batch count> batches.

## Findings
| # | Severity | File | Line | Finding | Recommendation |
|---|----------|------|------|---------|----------------|
<one row per finding; severities: critical, major, minor, nit. Keep the header when there are no findings.>

## Verdict
Verdict: approve
<or> Verdict: request-changes
<one sentence of rationale>
```

Do not modify source and do not call `github_pr_comment`; the report is local. If the gate rejects, repair the named gap and rewrite the whole report. Then call `advance_task_step`.

These deliverables live in the project's artifacts drawer — write them with `write_artifact` and read them back with `read_artifact`; the shipped workspace stays untouched. A pull-request review never modifies project source.

Deliver the verdictreviewer

The complete-coverage report passed its gates. Summarize the verdict and point the user to the local evidence files.

Show working prompt

Read the artifacts `{{workPath}}/pr-review.md` and `{{workPath}}/pr-review-coverage.json` with `read_artifact`, then write one final task note with `write_task_note`: `PR #{{number}} — Verdict: <approve|request-changes> — N findings (a critical, b major, c minor, d nit) — coverage X/X` plus a one-paragraph summary. Tell the user the full local review is at `{{workPath}}/pr-review.md` in the project's artifacts drawer, the coverage ledger is beside it at `{{workPath}}/pr-review-coverage.json`, the per-batch observations are under `{{workPath}}/pr-review/`, and nothing was posted to GitHub. Then call `advance_task_step` to complete the task.

Triggers

Phrases that suggest this craftbook to a crew.

review this pr
review the current pr
review this pull request
pr review
review the github pull request

Toolsets

[object Object]

Source

View this craftbook on GitHub · MIT license