稀疏注意力

Attention approximated by computing only a subset of token pairs.

Sparse attention skips unimportant pairs via local windows, global anchors or clustering, cutting long-sequence cost from quadratic to near-linear. It is a common accelerator in long-context models.