ArchitectureModel structures: Transformers, attention and mixture-of-experts
上下文学习
Learning a new task from examples in the prompt without updating weights.
In-context learning explains few-shot ability: models saw countless example-task patterns during pretraining and follow prompt examples at inference. It is why prompt engineering works.