Knowledge distillation
Knowledge distillation trains a student model using targets or distributions produced by a larger teacher. The goal is behavior transfer, not a literal compression that preserves every capability and failure mode.
Overview. Knowledge distillation trains a student model using targets or distributions produced by a larger teacher.
Context. The goal is behavior transfer, not a literal compression that preserves every capability and failure mode.
Claims and evidence
Claim ledger, 2Knowledge distillation trains a student model using targets or distributions produced by a larger teacher.
The goal is behavior transfer, not a literal compression that preserves every capability and failure mode.