Rethinking Pre-Training for Agentic AI with Aakanksha Chowdhery - #759

About this episode

Aakanksha Chowdhery discusses the need to rethink pre-training for agentic AI to enhance reasoning and planning capabilities.

Today, we're joined by Aakanksha Chowdhery, member of technical staff at Reflection, to explore the fundamental shifts required to build true agentic AI. While the industry has largely focused on post-training techniques to improve reasoning, Aakanksha draws on her experience leading pre-training efforts for Google’s PaLM and early Gemini models to argue that pre-training itself must be rethought to move beyond static benchmarks. We explore the limitations of next-token prediction for multi-step workflows and examine how attention mechanisms, loss objectives, and training data must evolve to support long-form reasoning and planning. Aakanksha shares insights on the difference between context retrieval and actual reasoning, the importance of "trajectory" training data, and why scaling remains essential for discovering emergent agentic capabilities like error recovery and dynamic tool learning. The complete show notes for this episode can be found at https://twimlai.com/go/759.

People in this episode

Host: Sam Charrington

Guest: Aakanksha Chowdhery

Topics covered

• agentic AI
• pre-training
• reasoning
• multi-step workflows
• attention mechanisms
• training data
• long-form reasoning

Keywords

• agentic AI
• pre-training
• reasoning
• multi-step workflows
• attention mechanisms
• trajectory training data
• dynamic tool learning

Mentioned in this episode

Organizations: Reflection, Google

Products: PaLM, Gemini

More episodes of The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

• Is RAG Dead? Lessons from Building AI for Tax Law with Alex Bowcut - #769 · June 9, 2026 · 52 min
• Relational Foundation Models for Enterprise Data with Jure Leskovec - #768 · May 21, 2026 · 1h 6m
• How to Find the Agent Failures Your Evals Miss with Scott Clark - #767 · May 7, 2026 · 53 min
• How to Engineer AI Inference Systems with Philip Kiely - #766 · April 30, 2026 · 55 min
• How Capital One Delivers Multi-Agent Systems with Rashmi Shetty - #765 · April 16, 2026 · 54 min
• The Race to Production-Grade Diffusion LLMs with Stefano Ermon - #764 · March 26, 2026 · 1h 3m