HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

About this episode

The episode discusses HERMES++, a unified driving world model that integrates 3D scene understanding and future geometry prediction.

🤗 Upvotes: 68 | cs.CV Authors: Xin Zhou, Dingkang Liang, Xiwu Chen, Feiyang Tan, Dingyuan Zhang, Hengshuang Zhao, Xiang Bai Title: HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation Arxiv: http://arxiv.org/abs/2604.28196v1 Abstract: Driving world models serve as a pivotal technology for autonomous driving by simulating environmental dynamics. However, existing approaches predominantly focus on future scene generation, often overlooking comprehensive 3D scene understanding. Conversely, while Large Language Models (LLMs) demonstrate impressive reasoning capabilities, they lack the capacity to predict future geometric evolution, creating a significant disparity between semantic interpretation and physical simulation. To bridge this gap, we propose HERMES++, a unified driving world model that integrates 3D scene understanding and future geometry prediction within a single framework. Our approach addresses the distinct requirements of these tasks through synergistic designs. First, a BEV representation consolidates multi-view spatial information into a structure compatible with LLMs. Second, we introduce LLM-enhanced world queries to facilitate…

People in this episode

Hosts: Jingwen Liang, Gengyu Wang

Topics covered

• 3D scene understanding
• autonomous driving
• future geometry prediction
• Large Language Models
• environmental dynamics

Keywords

• HERMES++
• 3D scene understanding
• autonomous driving
• Large Language Models
• future geometry prediction
• environmental dynamics

Mentioned in this episode

Organizations: Arxiv

Books & works: HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

More episodes of Daily Paper Cast

• EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments · June 13, 2026 · 24 min
• MiniMax Sparse Attention · June 13, 2026 · 26 min
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning · June 13, 2026 · 23 min
• InterleaveThinker: Reinforcing Agentic Interleaved Generation · June 13, 2026 · 21 min
• FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents · June 13, 2026 · 23 min
• Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding? · June 13, 2026 · 20 min