The Illusion of Thinking: What the Apple AI Paper Says About LLM Reasoning

About this episode

The episode discusses a new paper from Apple that critiques current evaluation methods for AI and presents new findings on Large Reasoning Models.

This week we discuss The Illusion of Thinking, a new paper from researchers at Apple that challenges today’s evaluation methods and introduces a new benchmark: synthetic puzzles with controllable complexity and clean logic. Their findings? Large Reasoning Models (LRMs) show surprising failure modes, including a complete collapse on high-complexity tasks and a decline in reasoning effort as problems get harder. Dylan and Parth dive into the paper's findings as well as the debate around ...

People in this episode

Hosts: Dylan, Parth

Topics covered

• AI
• Large Reasoning Models
• evaluation methods
• synthetic puzzles
• reasoning effort
• complexity

Keywords

• Apple AI
• LLM reasoning
• evaluation methods
• synthetic puzzles
• complexity
• reasoning effort
• failure modes

Mentioned in this episode

Organizations: Apple

More episodes of Deep Papers

• CUGA Agent: From Benchmarks to Business Impact of IBM's Generalist Agent · February 11, 2026 · 23 min
• TUMIX: Multi-Agent Test-Time Scaling with Tool-Use Mixture · November 24, 2025 · 24 min
• Meta AI Researcher Explains ARE and Gaia2: Scaling Up Agent Environments and Evaluations · November 10, 2025 · 23 min
• Georgia Tech's Santosh Vempala Explains Why Language Models Hallucinate, His Research With OpenAI · October 14, 2025 · 31 min
• Atropos Health’s Arjun Mukerji, PhD, Explains RWESummary: A Framework and Test for Choosing LLMs to Summarize Real-World Evidence (RWE) Studies · September 22, 2025 · 26 min
• Stan Miasnikov, Distinguished Engineer, AI/ML Architecture, Consumer Experience at Verizon Walks Us Through His New Paper · September 6, 2025 · 48 min