Data Quality Audit
Audit a dataset against explicit data-quality rules and produce a scorecard of where it passes and fails across completeness, validity, consistency, uniqueness, and timeliness. Defines the quality rules and dimensions FIRST (so 'quality' is measurable, not a vibe), then runs the audit to measure each rule with pass rates and failing examples, then writes a prioritized report with an overall score and remediation steps. Use this for a data quality audit, data health check, validation report, data-quality scorecard, or assessing whether a dataset is trustworthy.
Steps
Entry step: rules. Each step names the specialist role it wants; the full working prompt is expandable.
- Define quality rulesdata-analystentry
lock the dimensions, rules, and thresholds for 'good'
Show working prompt
Make data quality measurable before auditing. Step 1: choose the quality dimensions in scope — completeness (no missing required fields), validity (values match type/format/domain), consistency (cross-field logic, e.g. end_date >= start_date), uniqueness (no dupes on the key), timeliness (recency), and accuracy/referential integrity where checkable. Step 2: write concrete, testable rules per dimension naming the exact columns ('email matches an email regex', 'status in {active,churned,trial}', 'no duplicate customer_id'). Step 3: set a pass-rate threshold per rule (e.g. >= 99% non-null) and a weighting for the overall score. Step 4: write an acceptance-criteria checklist ('every dimension has at least one testable rule', 'each rule names its columns and threshold', 'audit reports a pass rate + failing examples per rule', 'an overall weighted score is produced'). Call write_task_note and write the rules + thresholds + weights + checklist to notes/rules.md. No auditing yet. - Run the auditdeveloper
measure every rule and emit pass rates with failing examples
Show working prompt
Run every quality rule against the full dataset deterministically. Step 1: for each rule, evaluate all rows and compute the pass rate (rows passing / total) and the count of failures. Step 2: capture up to 5 concrete failing examples per rule (the row id + the offending value) so the report can show evidence. Step 3: compare each pass rate to its threshold and mark the rule pass/fail. Step 4: compute the overall weighted quality score from the rule results and the weights. Step 5: emit data/audit.json — per-rule { dimension, rule, pass_rate, threshold, status, failing_examples } plus the overall score and a per-dimension rollup. Call write_task_note with the overall score and the worst rules. - Report the auditcopywriter
write a prioritized scorecard with remediation steps
Show working prompt
Write the data-quality report from the audit results — use only measured numbers. Step 1: open with the overall score and a one-line verdict on trustworthiness. Step 2: show a scorecard table: dimension, rule, pass rate, threshold, pass/fail. Step 3: detail the failing rules worst-first, each with its pass rate and 1-2 failing examples as evidence. Step 4: give a prioritized remediation list (fix the highest-impact, lowest-effort failures first) with a concrete action per failing rule. Step 5: note any rule that couldn't be evaluated and why. Write the report to report.md and call write_task_note with the path and overall score. On a loop-back, fix only the named gaps.
- Evaluatereviewer
Grade the deliverable against every acceptance criterion. All pass → finish; any fail → loop back and fix the gap.
Show working prompt
Read report.md and cross-check against notes/rules.md and data/audit.json. Verify: every quality dimension has at least one testable rule; each rule reports a pass rate against its threshold with a pass/fail; failing rules show concrete failing examples as evidence; an overall weighted score is reported; and remediation steps are prioritized and concrete. Re-compute one rule's pass rate against the audit data to confirm. Confirm no rule is marked passing while its examples show failures. Write PASS/FAIL per criterion; any FAIL loops back to report. Then route — this is the whole point of the loop: - **Every criterion PASSES →** call `advance_task_step({ ref, stepId: "evaluate", next: "finish" })`. - **Any criterion FAILS →** write the specific gaps to notes, then call `advance_task_step({ ref, stepId: "evaluate", next: "report" })` to loop back. The builder fixes exactly those gaps. Never route to `finish` while any criterion is unmet. The build phase's completion gate already blocked a grossly-incomplete deliverable; your job is the judgment an automated check cannot make (does it actually work, read well, look right). After ~3 unproductive loops, stop and report DONE_WITH_CONCERNS so the user can step in. - Finishdeveloper
All acceptance criteria met. Stamp a short summary and report DONE.
Show working prompt
Every acceptance criterion passed. Write a one-paragraph DONE summary to task notes via `write_task_note`: what was built, the deliverable path(s), and a one-line confirmation that each criterion is met. Then report DONE.
Triggers
Phrases that suggest this craftbook to a crew.
- audit data quality
- data health check
- is this data trustworthy
- data quality scorecard
- validate a dataset
Source
View this craftbook on GitHub · MIT license