Researchers compared vanilla reverse-KL on-policy distillation with GRPO across task difficulty, student and teacher size, domain, and optimisation choices, measuring held-out accuracy against training FLOPs. With an external post-trained teacher, OPD generally lost to GRPO on accuracy per train...
Source: [Dev.to](https://dev.to/paragonaimag/on-policy-distillation-loses-to-grpo-on-compute-efficiency-ba3)