24. Templates, prompts, and reusable artifacts
Use templates to make decisions inspectable, not to manufacture completeness. Delete fields that do not apply only with a reason. Leave unknowns explicitly marked rather than filling them with guesses.
- Use task, run, and decision templates.
- Record unknowns without inventing results.
- Connect conclusions to inspectable artifacts.
Use task, review, failure, and decision templates to preserve inspectable evidence. Unknowns stay unknown; completed fields should represent work that actually happened.
Task-contract template
Title and version:
Decision this eval informs:
Intended user and use:
System configuration:
Task unit and target population:
Input/output schema:
Allowed tools, data, and actions:
Reference standard and adjudication:
Development/holdout split and leakage controls:
Primary endpoint and denominator:
Secondary endpoints and critical failures:
Comparator and resource matching:
Exclusions and missing-output handling:
Proposed acceptance rule and accountable approver:
Data governance and publication restrictions:
Known limitations:Independent-review prompt
Review the supplied task contract, source, tests, and observed run.
Do not rely on the author's claimed verdict.
Identify concrete correctness, scope, security, and verification defects.
For each finding, cite the requirement and the exact artifact or
reproduction that supports it. Distinguish observed from hypothetical.
Do not penalize valid alternative implementations merely because they
differ from the reference patch. Report uncertainty explicitly.
Do not edit reference answers or execute unapproved external actions.Adapt this prompt to your employer’s approved review workflow.
Failure-analysis template
Case/run ID and artifact revisions:
Expected behavior and source/policy:
Observed behavior and exact evidence:
Reproduction:
Failure class and severity:
Competing explanations:
Discriminating checks performed:
Root cause supported by evidence:
Proposed repair:
Regression and fresh-case validation:
Remaining uncertainty:Decision-report template
Decision: advance / hold / revise experiment / reject tested use
Scope: system, task, users, and population
Execution: complete / incomplete, with errors and exclusions
Evidence: cohort, comparator, versions, primary outcome, uncertainty
Critical failures and subgroup findings:
Resource use: observed latency, tokens, cost; unknown where unavailable
Interpretation: what the result supports
Limits: what the result does not establish
Next action, owner, acceptance criteria, and rollback:A reusable agent instruction
“Preserve the task contract. Show executed evidence. Fail on missing inputs. Keep references out of model-visible material. Never change the denominator silently. Label synthetic outputs. Make the smallest complete repair. Report what remains unverified.”
Templates are also included in the starter kit so your report does not depend on copying a rendered web page.