InferenceInference performance, quantization and cost optimization

测试时计算

Extra compute allocated at inference time (search, verification, iteration).

Test-time compute is how inference-time scaling is implemented: models "think" longer, verifying multiple times before answering. Reasoning models like the o-series and DeepSeek-R1 rely on it.

Related terms