Co-Evolving Policy Distillation
From Daily Paper Cast by Jingwen Liang, Gengyu Wang
May 2, 2026 · 23 min · Episode 1827
About this episode
This episode discusses Co-Evolving Policy Distillation, a novel approach to integrating multiple expert capabilities in machine learning.
🤗 Upvotes: 34 | cs.LG Authors: Naibin Gu, Chenxu Yang, Qingyi Si, Chuanyu Qin, Dingyu Yao, Peng Fu, Zheng Lin, Weiping Wang, Nan Duan, Jiaqi Wang Title: Co-Evolving Policy Distillation Arxiv: http://arxiv.org/abs/2604.27083v1 Abstract: RLVR and OPD have become standard paradigms for post-training. We provide a unified analysis of these two paradigms in consolidating multiple expert capabilities into a single model, identifying capability loss in different ways: mixed RLVR suffers from inter-capability divergence cost, while the pipeline of first training experts and then performing OPD, though avoiding divergence, fails to fully absorb teacher capabilities due to large behavioral pattern gaps between teacher and student. We propose Co-Evolving Policy Distillation (CoPD), which encourages parallel training of experts and introduces OPD during each expert's ongoing RLVR training rather than after complete expert training, with experts serving as mutual teachers (making OPD bidirectional) to co-evolve. This enables more consistent behavioral patterns among experts while maintaining sufficient complementary knowledge throughout. Experiments validate that CoPD achieves all-in-one…
People in this episode
Hosts: Jingwen Liang, Gengyu Wang
Topics covered
- policy distillation
- reinforcement learning
- machine learning
- expert training
- model integration
Keywords
- CoPD
- policy distillation
- RLVR
- OPD
- machine learning
- expert capabilities
- training paradigms
More episodes of Daily Paper Cast
- EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments · June 13, 2026 · 24 min
- MiniMax Sparse Attention · June 13, 2026 · 26 min
- SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning · June 13, 2026 · 23 min
- InterleaveThinker: Reinforcing Agentic Interleaved Generation · June 13, 2026 · 21 min
- FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents · June 13, 2026 · 23 min
- Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding? · June 13, 2026 · 20 min
Explore listener stats, chart rankings, contacts and more on the Daily Paper Cast podcast page.