Shipping your first agent to production

Before flipping the agent on for users:

1. Golden set. 20–50 representative tasks with expected outcomes. Run before every prompt change.
2. Observability. Log model, tokens, latency, tool calls, errors. Dashboard the top 5.
3. Cost budget. Per-user and per-task hard limits. Refuse with a friendly message when hit.
4. Kill switch. A feature flag that turns the agent off without a deploy.
5. Human review queue. Sample 1–5% of completions and grade them weekly.
6. Rollout plan. Shadow → 1% → 10% → 100% with quality + cost gates at each step.

Agents that get production traction are the boring well-instrumented ones.