Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models

About this episode

The episode discusses the TIDE framework for cross-architecture distillation in diffusion large language models, highlighting its components and performance improvements.

🤗 Upvotes: 37 | cs.CL, cs.AI, cs.LG Authors: Gongbo Zhang, Wen Wang, Ye Tian, Li Yuan Title: Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models Arxiv: http://arxiv.org/abs/2604.26951v1 Abstract: Diffusion large language models (dLLMs) offer parallel decoding and bidirectional context, but state-of-the-art dLLMs require billions of parameters for competitive performance. While existing distillation methods for dLLMs reduce inference steps within a single architecture, none address cross-architecture knowledge transfer, in which the teacher and student differ in architecture, attention mechanism, and tokenizer. We present TIDE, the first framework for cross-architecture dLLM distillation, comprising three modular components: (1) TIDAL, which jointly modulates distillation strength across training progress and diffusion timestep to account for the teacher's noise-dependent reliability; (2) CompDemo, which enriches the teacher's context via complementary mask splitting to improve predictions under heavy masking; and (3) Reverse CALM, a cross-tokenizer objective that inverts chunk-level likelihood matching, yielding bounded gradients and dual-end…

People in this episode

Hosts: Jingwen Liang, Gengyu Wang

Topics covered

• cross-architecture distillation
• diffusion large language models
• knowledge transfer
• machine learning
• AI research

Keywords

• TIDE
• distillation
• dLLMs
• machine learning
• AI
• knowledge transfer
• performance benchmarks

Mentioned in this episode

Organizations: Arxiv

More episodes of Daily Paper Cast

• EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments · June 13, 2026 · 24 min
• MiniMax Sparse Attention · June 13, 2026 · 26 min
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning · June 13, 2026 · 23 min
• InterleaveThinker: Reinforcing Agentic Interleaved Generation · June 13, 2026 · 21 min
• FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents · June 13, 2026 · 23 min
• Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding? · June 13, 2026 · 20 min