InferenceInference performance, quantization and cost optimization

流式输出

Returning tokens as they are generated, so users need not wait for the full answer.

Streaming pushes tokens via SSE-style protocols, bringing the first token in hundreds of milliseconds and greatly improving chat UX; most APIs support it. Billing still counts full output tokens.

Related terms