InferenceInference performance, quantization and cost optimization
推理(模型推理)
Using a trained model to produce output for new input.
Inference is the "run phase": the user sends a prompt and the model generates tokens. Cost is billed per token; optimization covers quantization, KV cache, batching and test-time compute.