01
Secure the boundary
Assume every prompt, tool argument, document, and model output is untrusted.
Authenticate the user, authorize every data query by tenant and user, and keep provider credentials on the server. Define tool capabilities as allowlists, redact sensitive values from traces, and treat retrieved documents as data rather than instructions. Add rate limits to expensive and state-changing paths.
- Enforce server-side userId or tenantId scoping on every user-data query.
- Use short-lived credentials and least privilege.
- Test prompt injection through retrieved content and tool responses.
Security is a property of the whole execution path, not a prompt appended at the end.
02
Design for failure
Production reliability comes from bounded degradation, not perfect model behavior.
Set timeouts at every network boundary and propagate a request deadline. Make writes idempotent, retry only transient failures, and use a queue for work that outlives the request. Provide deterministic fallbacks for common tasks and a human escalation path for ambiguous or high-impact actions.
- Separate synchronous user feedback from durable background work.
- Use circuit breakers for failing providers and tools.
- Persist checkpoints so deploys and restarts do not erase progress.
Every dependency needs a timeout, an owner, a fallback, and a way to explain failure.
03
Measure before you scale
You cannot improve an agent you cannot inspect.
Track task success, tool error rate, latency, token usage, cost per successful task, escalation rate, and user correction rate. Slice metrics by model, workflow, tenant, and version. Establish a rollback threshold before launch and review traces weekly during the first month.
- Create dashboards for both technical and outcome metrics.
- Version prompts, tools, models, and evaluation datasets.
- Set spend alerts and per-request budgets before opening access.
Launch with a narrow audience, a rollback plan, and enough telemetry to know whether the system is helping.
Ship the smallest reliable workflow first. Add autonomy only when your evaluation data shows that it improves outcomes.