Streaming guide

Treat a streamed response as an interruptible sequence

The client can disconnect, the model can stop early and tool arguments can arrive in parts. Build the UI and server path around partial state.

Create account Read the docs View pricing

Planned recharge methods: Alipay and USDT. Collection is not enabled yet.

Handle these cases

First-token delay
Client cancellation
Partial text
Partial tool arguments
Network interruption
Final usage and finish reason

Check model support

Use streaming only when the live catalog marks the selected model as supporting it. Keep a non-streaming fallback for diagnostics.

Common questions

Does every model support streaming?

No. Check the capability flag in the live catalog.

Should I save partial output?

That depends on the product. For auditable workflows, distinguish incomplete output from a completed model response.

Related resources

Official Catalog of Available LLM Models Retry only errors that are safe to repeat Function calling is a proposal, not permission to execute Use the OpenAI SDK by changing three configuration values