Typical categories
- Authentication: fix the key or account state; do not retry blindly.
- Validation: correct the request.
- Rate limit: back off with jitter.
- Timeout: retry only when the operation is idempotent.
- Upstream unavailable: fail over only to a tested model path.
Log useful evidence
- Request ID
- Model name
- HTTP status
- Error code
- Attempt count
- Duration
Never log the full API key or sensitive prompt data by default.
Common questions
How many times should I retry?
Use a small bounded number based on the error class and request latency budget. There is no safe universal unlimited retry policy.
Can I retry a tool action?
Only when the action is idempotent or protected by an idempotency key and your application can determine the previous outcome.