直接偏好优化（DPO）

A simplified alignment method that optimizes the policy directly from preference data, without a reward model.

DPO merges reward modeling and RL into one step, deriving a loss from preference pairs. Simple to implement and stable to train, it has become one of the most popular alignment methods in the open-source community.