ArchitectureModel structures: Transformers, attention and mixture-of-experts

上下文学习

Learning a new task from examples in the prompt without updating weights.

In-context learning explains few-shot ability: models saw countless example-task patterns during pretraining and follow prompt examples at inference. It is why prompt engineering works.

Related terms