ArchitectureModel structures: Transformers, attention and mixture-of-experts
编码器-解码器
A two-part architecture where an encoder reads the input and a decoder generates output.
The encoder compresses input into a semantic representation; the decoder generates tokens from it. T5 and BART use this; mainstream LLMs are mostly decoder-only, merging prompt and continuation in one pass.