Appendix - Production Agents
What We Learned
- Separate orchestration, models, tools, state, review and telemetry; enforce policy outside the model.
- Constrain data access, credentials and outbound actions to limit injection impact.
- Persist execution and make side effects idempotent so retries and pauses are recoverable.
- Track cost per successful task, latency and budgets across the whole run.
- Combine regression, capability and safety evals with traces; apply review boundaries in the system designs.
Key Acronyms, Concepts & Jargon
| Term | Short meaning |
|---|---|
| Durable execution | Persists progress so a run can resume after failures or long waits. |
| Replay / checkpoint | Rebuild state from recorded events / restore a saved execution state. |
| Idempotency key | Identifies an operation so retries do not duplicate its intended effect. |
| Circuit breaker | Temporarily blocks calls to a repeatedly failing dependency. |
| HITL | Human In The Loop: review or approval at selected decision boundaries. |
| Lethal trifecta | Private data, untrusted input and outbound communication in one agent context. |
| Sandbox / least privilege | Restricted execution environment / minimum necessary permissions. |
| Prompt injection / memory poisoning | Redirecting instructions through input / planting harmful persistent context. |
| OTel / trace / span | OpenTelemetry / end-to-end run record / timed operation within that record. |
| Regression / capability / safety suite | Protect known behavior / measure new ability / test unacceptable behavior. |
| pass^k | Repeated-run reliability: success on all k trials. |
| Cost per successful task | Total evaluated cost divided by successful task completions. |
| PII / PHI | Personally Identifiable Information / Protected Health Information. |
| PA / audit trail | Prior Authorization / record of actions, decisions and responsible identities. |