#129 - Your Screen Is the Training Data: AI Now Learns Your Job by Watching You Work

About this episode

This episode discusses how AI learns from user actions to automate tasks, highlighting recent developments from major AI labs.

Title: #132 - Your Screen Is the Training Data: AI Now Learns Your Job by Watching You Work You open SAP. You copy a number into Excel. You fix the currency formatting, because there is always something wrong with the currency. You hit submit. You have done it a thousand times — you could explain it in your sleep. Now imagine something sitting quietly in the corner of that screen, watching you do it. Not to grade you. To learn it — so the next thousand times, it can do it without you. That is not a thought experiment anymore. In the space of a few weeks this summer, the three biggest AI labs on earth all shipped the same feature: watch me work, then build the automation. And the uncomfortable part is not that you will want to use it. It is that your management may decide everyone should. 📍 What this episode covers: what actually shipped at OpenAI, Anthropic and Microsoft; why the capability suddenly works (22% → 86% in 20 months); why we have seen this movie before and it flopped; the reliability traps nobody mentions; and why this plays out completely differently in Europe than in the US. 🧰 Three labs, one feature, one summer. OpenAI shipped Record & Replay on 22 June —…

People in this episode

Host: Malcolm Werchota

Topics covered

• AI automation
• workflow learning
• machine learning
• productivity tools
• technology trends

Keywords

• AI
• automation
• workflow
• OpenAI
• Anthropic
• Microsoft
• Record & Replay
• Computer History
• productivity

Mentioned in this episode

Organizations: OpenAI, Anthropic, Microsoft

Products: Record & Replay, Computer History

More episodes of The AI Cookbook Show by Malcolm Werchota

• #132 - The Mathematician Who Beat OpenAI by Three Days. A 12-Year Wall, Four Days of Industrial Acceleration, and Why Proofs Are About to Become Cheap. · September 9, 2026 · 41 min
• #131 - [AI DRAMA] - The AI Conspiracy That Actually Happened. 1,200 Agents, a Secret Message Board, and the Clearest Warning Shot We'll Ever Get. · September 5, 2026 · 43 min
• #130 - The Harness Is The Work: Why The Chef Isn't The Point · August 24, 2026 · 32 min
• #128 - How ChatGPT Cracked an 80-Year-Old Math Problem for $1,000 · June 11, 2026 · 27 min
• #127 - [Quickbite] - Chief AI Academy — Sneak Peek into Session 1 · June 8, 2026 · 21 min
• #126 - [AI Drama] - Your Continuity Plan Doesn't Cover Drones. It Should. · June 5, 2026 · 56 min