Reliability & Resilience Review
Review a codebase for production reliability and resilience risks across asynchronous work, external dependencies, state transitions, resource lifecycles, and recovery paths. Uses the repository and per-file indexes to map failure domains, verifies likely issues in source and tests, and leaves a prioritized morning report covering timeouts, retries, idempotency, cancellation, concurrency, backpressure, cleanup, graceful shutdown, observability, and disaster recovery. Use for a weekly reliability sweep, resilience review, failure-mode audit, or pre-production hardening review.
Steps
Entry step: model. Each step names the specialist role it wants; the full working prompt is expandable.
- Map failure domainsreviewerentry
inventory stateful and asynchronous boundaries plus their reliability contract
Show working prompt
Call `map_repo`, `list_file_issues`, and targeted `search_code` queries for queues/jobs, timers, promises, subprocesses, network/database/filesystem calls, caches, locks, transactions, startup/shutdown, and persistence. Build a failure-domain map: critical user journeys, state owners, external dependencies, retry boundaries, resource lifecycles, and recovery expectations. For each boundary define the reliability contract: timeout, retry/backoff/jitter, idempotency, cancellation, concurrency limit/backpressure, cleanup, error propagation, monitoring, and recovery. Record index coverage and areas not statically observable. Write the map and prioritized audit plan to `notes/reliability/failure-map.md`.
- Verify failure handlingreviewer
trace representative failures through source and tests
Show working prompt
For each high-impact boundary, inspect the relevant symbols and callers with `file_review`, `outline_file`, `read_symbol`, and `find_references`. Trace representative failures: slow/hung dependency, transient failure, duplicate delivery, partial write, cancellation, concurrent mutation, process restart, disk full/corrupt state, and shutdown during work. Locate tests and run safe existing checks when practical. Confirm whether timeouts are bounded; retries have caps/backoff/jitter and do not multiply side effects; operations are idempotent; cancellations and errors propagate; resources/timers/listeners/subprocesses are released; transactions or durable journals protect multi-step state; startup and shutdown are safe; and operators get useful signals. Record confirmed issues and verified-safe controls with real file:line/test/command evidence in `notes/reliability/evidence.md`.
- Write the reliability reportreviewer
ranked failure-mode report with concrete hardening work
Show working prompt
Write `reliability-review.md` with `## Verdict`, `## Coverage and assumptions`, `## Failure-domain scorecard`, `## Findings`, `## Verified controls`, `## Not verified`, and `## Hardening sequence`. Each finding must cite real file:line or test/command evidence, state the initiating failure, propagation path, user/operator impact, likelihood, severity, and a concrete remediation plus the failure-injection/regression test to add. Rank by impact × likelihood × recoverability. Distinguish a missing control from a merely untested control. Do not demand distributed-systems machinery for local, low-risk code. On a loop-back, fix only named gaps.
- Evaluatereviewer
Grade the deliverable against every acceptance criterion. All pass → finish; any fail → loop back and fix the gap.
Show working prompt
Grade `reliability-review.md`: (1) critical failure domains and contracts are mapped; (2) findings cite real source/test/command evidence; (3) timeouts, retries/backoff, idempotency, cancellation, concurrency/backpressure, cleanup, partial writes, restart, and shutdown were considered where relevant; (4) propagation and user/operator impact are explained; (5) each remediation includes a failure-injection or regression test; (6) verified controls and unverified areas are explicit; (7) priorities reflect impact, likelihood, and recoverability without overengineering. PASS all or loop back. Then route — this is the whole point of the loop: - **Every criterion PASSES →** call `advance_task_step({ ref, stepId: "evaluate", next: "finish" })`. - **Any criterion FAILS →** write the specific gaps to notes, then call `advance_task_step({ ref, stepId: "evaluate", next: "report" })` to loop back. The builder fixes exactly those gaps. Never route to `finish` while any criterion is unmet. The build phase's completion gate already blocked a grossly-incomplete deliverable; your job is the judgment an automated check cannot make (does it actually work, read well, look right). After ~3 unproductive loops, stop and report DONE_WITH_CONCERNS so the user can step in. - Finishdeveloper
All acceptance criteria met. Stamp a short summary and report DONE.
Show working prompt
Every acceptance criterion passed. Write a one-paragraph DONE summary to task notes via `write_task_note`: what was built, the deliverable path(s), and a one-line confirmation that each criterion is met. Then report DONE.
Triggers
Phrases that suggest this craftbook to a crew.
- reliability review
- resilience audit
- review failure modes
- weekly production hardening
- find reliability risks
- nightly reliability sweep
Toolsets
- [object Object]
- [object Object]
Source
View this craftbook on GitHub · MIT license