22. An apprenticeship plan and portfolio
Learn through progressively more demanding artifacts. The point is not to finish a fixed number of days; it is to meet acceptance criteria that expose whether you understand the work. Repetition on real repositories develops the engineering judgment that tools cannot confer automatically.
- Build portfolio evidence through tested artifacts.
- State exactly what each project proves.
- Choose the next missing skill from observed failures.
Build a portfolio through increasingly demanding artifacts, not self-awarded titles. A small reproducible project with honest limits is stronger than an impressive but unverifiable demo.
Stage 1: the measurement loop
Run the kit, test failure paths, export blinded prompts, and inspect the report. Deliver a task contract and one evidence-based failure analysis. You pass this stage when you can explain the scorer without asking an agent to translate every line.
Stage 2: coding review
Complete the normalization repair lab. Preserve before-and-after behavior, exact diff, regression tests, and the agent trace. Then create a second small bug with a different mechanism, such as duplicate identifiers or missing null handling, and evaluate another repair.
You pass when you distinguish a valid alternative implementation from a grader defect and can explain why the tests cover the intended behavior.
Stage 3: domain reference standards
Develop a small synthetic extraction or evidence-classification cohort. Document labels, ambiguity policy, and adjudication. Recruit qualified independent review when the project becomes a substantive domain study. Do not invent reviewer participation for a portfolio.
You pass when labels are defensible from the supplied evidence, and the report separates task performance from downstream clinical or operational impact.
Stage 4: real system comparison
With an approved budget, run two declared configurations on matched cases. Preserve raw outputs and manifest details. Inspect discordant cases and compare resources as well as performance. Use a frozen acceptance set after development.
You pass when another reviewer can reproduce the analysis and understand which differences are observed and which remain uncertain.
Stage 5: production reasoning
Design and test isolation, cancellation, retries, logging, and data governance. Evaluate a tool-using workflow with explicit denied actions and partial failures. Review architecture decisions against requirements rather than style.
You pass when failure recovery is demonstrable and no incomplete run masquerades as success.
Three credible portfolio projects
A coding-agent review packet demonstrates engineering evaluation. A medication-extraction study demonstrates clinical reference-standard work. A temporally faithful surveillance backtest demonstrates epidemiological evaluation. Keep them small enough to review deeply. Publish only permitted synthetic or appropriately governed material.
For each include the question, dataset provenance, task contract, reference policy, implementation, tests, observed results, failure taxonomy, and limitations. A polished website without inspectable evidence is a weaker portfolio than a modest project with a trustworthy report.