注意力机制

A mechanism that lets a model dynamically focus on important parts of the input.

Attention computes relevance weights between every pair of input units, letting the model combine information by relevance. It is the core of Transformer, often described as a query-key-value matching process.