Squeezing AI into your Pocket

About this episode

This episode discusses the transition of language models from cloud to local devices by 2026 and the engineering advancements that made it possible.

Send us Fan Mail By 2026, language models have moved off the cloud and onto the device in your pocket. What was a research demonstration two years ago is now a routine engineering capability, and the centre of gravity for artificial intelligence has begun to migrate from distant data centres to local silicon. The episode traces the four engineering moves that made this possible. Quantization, which shrinks a model by storing its parameters with less precision. Optimized key-value caches, whic...

People in this episode

Host: David Such

Topics covered

• AI
• language models
• engineering
• local silicon
• quantization

Keywords

• AI
• language models
• quantization
• engineering
• local devices

More episodes of Embedded AI - Intelligence at the Deep Edge

• Who is Liable for Onboard AI? · May 31, 2026 · 24 min
• A chip that controls a balancing propeller on seven microwatts · May 14, 2026 · 15 min
• Why 95% of AI Deployments Fail · May 7, 2026 · 23 min
• Why Humans and Robots must Dream · April 25, 2026 · 24 min
• Sovereign AI and the End of the Borderless Cloud · April 19, 2026 · 20 min
• The Agentic AI Reckoning: Autonomy, Safety, and the Edge · April 13, 2026 · 25 min