Direct Preference Optimization
Direct Preference Optimization trains from preference pairs without fitting a separate explicit reward model. Its objective relates preference likelihood to a reference policy through a classification-style loss.
Overview. Direct Preference Optimization trains from preference pairs without fitting a separate explicit reward model.
Context. Its objective relates preference likelihood to a reference policy through a classification-style loss.
Claims and evidence
Claim ledger, 2Direct Preference Optimization trains from preference pairs without fitting a separate explicit reward model.
Its objective relates preference likelihood to a reference policy through a classification-style loss.