Use layered limits
- Project budget for the environment
- Key budget for the integration
- RPM for request bursts
- TPM for token-heavy traffic
- Concurrency for resource pressure
- Application loop limits for agents
Return a useful user experience
When a limit is reached, surface a clear message and a retry time where applicable. Do not silently switch to an untested model just to keep a request moving.
Common questions
Is RPM enough for AI agents?
No. An agent can produce large requests or long outputs within a modest request count. Combine RPM with TPM, concurrency, budgets and loop limits.
Where should limits be set?
Set broad limits at project level and narrower limits on each key according to the integration's purpose.