Interview and launch

Open honestly: models can't reliably separate instructions from data, so injection is mitigated, not solved.

Then layers: treat all retrieved and tool content as untrusted and delimit it; least privilege and scoped credentials; human approval for side effects; don't combine private data, untrusted input and an exfiltration path in one agent; output filtering (URL allow-lists, no auto-rendered images, schema validation); input and output classifiers; monitoring and an injection eval set in CI.

Close with your Flagship 1 example: the planted-instruction test and what it caught.

Measure: cost by feature, model, prompt section and user; find the top drivers (usually long repeated prompts, agent loops, or using a frontier model for easy traffic).

Levers: prompt caching for stable prefixes; trim context (fewer, reranked chunks; compaction; truncated tool results); route easy requests to smaller models; semantic and exact caching; batch APIs for offline work; cap output length; distil or fine-tune a small model for a high-volume narrow task; self-host at very high volume.

Guard: run the eval suite on every change so quality is proven unchanged, and roll out gradually.

Be ready to explain: data leakage with concrete examples and the pipeline fix; why accuracy fails under imbalance and what to use; precision vs recall in terms of business cost; bias–variance and how to diagnose it with learning curves; why gradient boosting dominates tabular data; time-series splitting.

Interviewers use these to check whether your LLM evaluation instincts rest on solid foundations.

Revisit the week 6–11 quizzes and aim for full marks without notes.

Do at least three mocks: one system design (RAG at scale, an agent platform, an eval pipeline), one coding (Python, data manipulation, a small LLM integration), one project deep-dive (expect 'what would you do differently?' and 'how do you know it works?').

Use a structure for design: requirements → estimates → high-level design → deep dive on the riskiest component → evals, cost and failure modes → trade-offs.

Record yourself or practise with a peer. Fix the two weakest answers before the real ones.

Track every application: company, role, date, contact, stage, next action, notes on what was asked. Review weekly; patterns in rejections tell you what to fix. Referrals and direct outreach to engineers beat cold applications.

After the path: pick three ongoing inputs (not twenty), such as Simon Willison's blog, Hugging Face daily papers and one newsletter. Read one paper a month, and ship a small project or post every month or two.

The field moves fast, but the foundations you built (evaluation discipline, attention, retrieval, production engineering) change slowly. That is what lasts.