← All craftbooks

CI Pipeline from Scratch

Build: software and codeevalv2.0.2released 2026-09-05workflow: build-loop

Author a CI pipeline grounded in what the repo actually has: inventory the real package scripts, write the workflow at the conventional location with caching and merge-gating on green, and verify every command it invokes against the manifests — honest that CI itself cannot run locally. Adds a publish-on-tag release job only when the repo shows one.

Steps

Entry step: inventory. Each step names the specialist role it wants; the full working prompt is expandable.

  1. Inventory the repodevops engineerentry

    Read the real scripts and layout, then plan stages, caching, and triggers — never a command that does not exist.

    Show working prompt
    Learn what this repo actually needs before writing a line of YAML. Read `package.json` (and any workspace manifests) with `read_file` and record the scripts that genuinely exist — their exact names and the exact lines. Read the repo layout with `list_dir`: the lockfile (which package manager), the language, build outputs, and any existing workflow files. NEVER invent a command: a stage enters your plan only when a matching script or standard invocation exists in the repo — install, lint, typecheck, test, build, in whatever subset is real here, and nothing for tools the repo does not use. Decide caching from the lockfile and package manager, and the triggers the task calls for (typically pushes to the default branch and pull requests). Plan a release or publish stage ONLY when the repo shows one — a publish configuration, a release script, an existing tag convention; when nothing shows one, say so and leave it out. Note the CI host the repo points at (existing workflow files, hosting hints); default to GitHub Actions when nothing says otherwise.
    
    Write `{{workPath}}/ci-plan.md` with exactly these sections: `## Detected commands` (each script name with the exact line quoted from `package.json`), `## Pipeline stages` (the stages in order and why each exists here), `## Caching` (what is cached and what it is keyed on), `## Triggers` (the events and why). Cite the real files with backtick paths.
    
    Observable handoff: write the completed result to `{{workPath}}/ci-plan.md` in the project's artifacts drawer with `write_artifact`. Do not merely describe what the file would contain. Re-read it with `read_artifact` before finishing this phase and repair any incomplete sections.
  2. Author the workflowdevops engineer

    Write the real workflow file(s) at the conventional location — minimal, cached, pinned, gating merges on green.

    Show working prompt
    Read `{{workPath}}/ci-plan.md` and write the REAL workflow file(s) with `write_file` at the conventional location for the detected host — for GitHub Actions, `.github/workflows/ci.yml` unless the repo already has its own convention. Keep it minimal and fast: the planned stages in order, dependency caching wired to the lockfile, and the workflow triggering on the planned pushes and pull requests so merges gate on green. Pin every action to a major version (for example `actions/checkout@v4`); do not invent action names, inputs, or secrets. Every `run:` line must invoke a command the plan detected — nothing else. Add a release or publish job ONLY if ci-plan.md called for one, and gate it behind the green build/test stages and the tag trigger so a publish can never run before the checks pass; if the plan named no release stage, do not add one.
    
    Then write `{{workPath}}/ci-notes.md` with exactly these sections: `## Files written` (the real path of every workflow file, backtick path per line), `## Stages` (each job and step, and the detected command it runs), `## Choices explained` (caching, triggers, pinning, and anything you deliberately left out — including a release job when the repo showed no evidence for one). Before working, open `{{workPath}}/ci-plan.md` with `read_artifact`.
    
    Observable handoff: write the completed result to `{{workPath}}/ci-notes.md` in the project's artifacts drawer with `write_artifact`. Do not merely describe what the file would contain. Re-read it with `read_artifact` before finishing this phase and repair any incomplete sections.
  3. Verify against the manifestsdevops engineer

    Read the workflow back and match every invoked command to the manifest line that provides it; honest about what only a live run proves.

    Show working prompt
    Verify the workflow the only honest way available without a CI run: read it back and check every claim against the repo. Open each workflow file named in `{{workPath}}/ci-notes.md` with `read_file`, and for EVERY command it invokes, find and quote the `package.json` line (or manifest entry) that provides it — a `run:` line with no matching script is a defect: fix it now at the workflow's real site, not in prose. Confirm the triggers match `{{workPath}}/ci-plan.md`. Confirm every action reference is pinned to a major version, and every secret the workflow references is a standard one (`GITHUB_TOKEN`) or one the user explicitly named — an invented secret is a defect. Be plain about the limit of this verification: CI cannot be executed locally, so state what reading verified and what only a real run on the host can prove — never claim the pipeline ran.
    
    Write `{{workPath}}/verification.md` with exactly these sections: `## Workflow files` (each file checked, backtick paths), `## Commands verified against package.json` (each workflow command with the quoted manifest line that provides it), `## Result` (what reading verified, what only a live CI run can prove, and any defects found and fixed). Before working, open `{{workPath}}/ci-plan.md`, `{{workPath}}/ci-notes.md` with `read_artifact`.
    
    Observable handoff: write the completed result to `{{workPath}}/verification.md` in the project's artifacts drawer with `write_artifact`. Do not merely describe what the file would contain. Re-read it with `read_artifact` before finishing this phase and repair any incomplete sections.
  4. Evaluate the deliverabledevops reviewer

    Independently grade the observable deliverable and route it to finish, repair, or user escalation.

    Show working prompt
    Review `{{workPath}}/verification.md`, `{{workPath}}/ci-plan.md`, `{{workPath}}/ci-notes.md` against every criterion below. Inspect the underlying evidence files named by the workflow; do not grade from the author's summary alone.
    
    1. Every command the workflow invokes exists in the repo — open package.json (and any other manifests) and check each run: line yourself; a script the repo does not have fails this criterion.
    2. The stages are ordered sensibly (dependencies installed before lint/test/build) and dependency caching is wired to the real lockfile.
    3. The triggers match ci-plan.md, and a release or publish job exists only where the plan showed repo evidence for one — gated behind the green stages and the tag trigger when present.
    4. Nothing is invented: every action is pinned to a major version, no fabricated action inputs, and no secrets beyond standard ones or those the user explicitly named.
    5. verification.md is honest about the limit of local verification — what reading proved versus what only a live CI run can prove — and the accounts reconcile across ci-plan.md, ci-notes.md, and verification.md.
    
    Open `{{workPath}}/verification.md`, `{{workPath}}/ci-plan.md`, `{{workPath}}/ci-notes.md` with `read_artifact`. Write the evidence-backed review to `{{workPath}}/review.md` in the artifacts drawer with `write_artifact`.
    
    List the findings as a markdown table with columns `| Severity | File | Line | Problem | Fix |` (severities: critical/major/minor/nit; empty table only on PASS). Give each criterion a PASS or FAIL with a concrete path, excerpt, measurement, or observed behavior. End with exactly `Verdict: PASS` or `Verdict: REVISE`. The gate ENFORCES the verdict: a well-formed REVISE is rejected and routed back to `repair` automatically, carrying your findings — so list every finding in the table with a concrete fix. On PASS, `advance_task_step` to `finish`. Never write PASS while a criterion is unmet.
  5. Finishproject lead

    All deterministic and reviewer criteria passed.

    Show working prompt
    The independent review passed. Read `{{workPath}}/review.md` with `read_artifact`, then use `write_task_note` to record a concise DONE summary with the final deliverable paths (`{{workPath}}/verification.md`, `{{workPath}}/ci-plan.md`, `{{workPath}}/ci-notes.md`) and the evidence that each acceptance criterion passed. Report DONE without starting new work.
  6. Repair the deliverabledevops engineer

    Fix only the concrete gaps from the latest independent review.

    Show working prompt
    Read `{{workPath}}/review.md` with `read_artifact` and repair every failed criterion in `{{workPath}}/verification.md`, `{{workPath}}/ci-plan.md`, `{{workPath}}/ci-notes.md`. Make changes on each file's declared surface (`write_artifact` for artifact inputs, `write_file` for workspace inputs), not in task notes or a reply. Preserve evidence that already passed. Re-run or re-check anything the reviewer found unproven. Ensure `{{workPath}}/verification.md` is genuinely updated this turn so the repair is observable, then hand it back for independent evaluation.
  7. Escalate unresolved concernsproject lead

    The bounded repair loop ended without a defensible pass.

    Show working prompt
    The deliverable did not pass after 3 review rounds. Do not claim success. Read `{{workPath}}/review.md` with `read_artifact`, then use `write_task_note` to record DONE_WITH_CONCERNS: the unmet criteria, what was attempted, the affected paths, and the smallest user decision or missing input needed to continue.

Triggers

Phrases that suggest this craftbook to a crew.

Source

View this craftbook on GitHub · MIT license