Direct Preference Optimization
DPO derives a direct preference-learning objective from the relationship between a policy and implicit reward. The method avoids training a separate reward model and running reinforcement-learning optimization in its core pipeline.
Overview. DPO derives a direct preference-learning objective from the relationship between a policy and implicit reward.
Context. The method avoids training a separate reward model and running reinforcement-learning optimization in its core pipeline.
Claims and evidence
Claim ledger, 2DPO derives a direct preference-learning objective from the relationship between a policy and implicit reward.
The method avoids training a separate reward model and running reinforcement-learning optimization in its core pipeline.