"The Cognitive Revolution"

GovAI Research Fellow Alan Chan joins Nathan to examine the governance and safety infrastructure required for autonomous AI agents, spanning real-time monitoring, agent identifiers, and the risks of automating AI R&D. Reviewing recent rogue agent incidents and progress since his foundational papers, Chan details why the absence of synchronous monitoring during training and testing poses an immediate security vulnerability. The discussion tackles the lack of traceable agent IDs across the open web, the limits of chain-of-thought interpretability, and the necessity of concrete triggers to halt systems that breach their sandboxes. Together, they outline the technical commitments frontier labs must make to prevent loss-of-control events as automated systems accelerate toward an intelligence explosion. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/governing-ai-agents-alan-chan-of-govai-on-monitoring-agent-ids-rogue-agents-preparing-for-rsi/ Sponsors: OutSystems: OutSystems is the leading agentic systems platform, helping enterprises build, modernize, and operate mission-critical applications at the speed of AI. Learn more and start owning your agentic future at https://outsystems.com/tcr Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr Tasklet: Tasklet empowers your business with AI agents that connect to your tools and automate recurring workflows with no code required. Visit https://tasklet.ai and use code cog rev for $50 in free credits CHAPTERS: (00:00) About the Episode (05:10) Monitoring AI agents (17:50) Logging and agent identity (Part 1) (18:04) Sponsors: OutSystems | Athena (21:29) Logging and agent identity (Part 2) (31:58) Gatekeeping agent internet access (Part 1) (36:17) Sponsors: Claude | Tasklet (39:29) Gatekeeping agent internet access (Part 2) (49:12) Capitalizing and insuring agents (58:11) Rise of rogue agents (01:10:14) Assessing AI takeover risks (01:21:18) Intelligence explosion paper (01:31:18) Measuring AI R&D (01:46:04) Pacing the frontier (01:58:54) Designing autonomous AI auditors (02:04:40) AI whistleblowers and swarms (02:13:40) Episode Outro (02:16:59) Outro PRODUCED BY: https://aipodcast.ing

Nathan Labenz reports back from The Curve conference, sharing off-the-record insights from frontier lab leaders on shortened timelines and whether humanity is approaching an intelligence threshold it should not cross. The episode also features discussions with Positron CTO Thomas Sohmers on overcoming the memory wall while token spend eclipses human salaries, and swyx on how coding agents are disrupting enterprise software. In addition, Atlas Ignota's Evan Miyazono examines unowned risks from accidental agent swarms hitting critical infrastructure, while Mercor's Edward Hu explores the future of training data. Together, these conversations highlight how escalating compute demands and autonomous agent workflows are challenging long-held assumptions about hardware, software economics, and safety limits. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/ai-am-a-level-we-shouldn-t-pass-notes-from-the-curve-tokens-vs-salaries-is-saas-cooked/ Sponsors: Parallel: Parallel provides enterprise-grade web search APIs for AI agents, offering the optimal balance of quality, speed, and cost. Get started for free at https://parallel.ai/tcr Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr OutSystems: OutSystems is the leading agentic systems platform, empowering enterprises to build, coordinate, and govern AI agents and mission-critical applications securely. Learn more and start building your agentic future at https://outsystems.com/tcr Tasklet: Tasklet empowers your business with AI agents that connect to your tools and automate recurring workflows with no code required. Visit https://tasklet.ai and use code cog rev for $50 in free credits CHAPTERS: (00:00) Weekly highlights preview (01:57) Notes from The Curve (Part 1) (14:22) Sponsors: Parallel | Claude (17:17) Notes from The Curve (Part 2) (17:17) Timelines and compute caps (Part 1) (28:40) Sponsors: OutSystems | Tasklet (31:36) Timelines and compute caps (Part 2) (31:37) The memory wall (40:50) AI in chip design (48:17) Software after agents (54:33) Kill my SaaS (01:03:32) Who owns AI risks (01:10:58) Workplace training data (01:17:36) Beyond verifiable tasks (01:20:56) Episode Outro (01:25:58) Outro PRODUCED BY: https://aipodcast.ing

Keerthana Gopalakrishnan, Research Lead for Gemini Robotics at Google DeepMind, returns to discuss the release of Gemini Robotics 2, whole-body intelligence, and the pursuit of generalist humanoid robots. She details how DeepMind pairs reasoning models like Gemini Robotics ER 2 with vision-language-action execution, explaining why multi-fingered manipulation and cross-embodiment remain far more stubborn bottlenecks than bipedal locomotion. Together, they examine the real-world stakes of bringing physical AI into human environments, addressing crucial trade-offs across inference latency, sensor failures, and operational safety where errors carry immediate physical consequences. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/one-brain-any-body-google-deepmind-s-keerthana-on-gemini-robotics-2-cross-embodiment-humanoids/ Sponsors: Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive Parallel: Parallel provides enterprise-grade web search APIs for AI agents, offering the optimal balance of quality, speed, and cost. Get started for free at https://parallel.ai/tcr Deepgram Flux TTS: Deepgram Flux TTS is a streaming text-to-speech model built for voice agents with natural tone, context awareness, and interruption handling. Try it free until September 12 at https://deepgram.com/keep-talking OutSystems: OutSystems is the leading agentic systems platform, helping enterprises build, modernize, and operate mission-critical applications at the speed of AI. Learn more and start owning your agentic future at https://outsystems.com/tcr Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (04:22) Humanoid Olympics and simulation (11:11) Generalization versus task mastery (21:22) Gemini Robotics 2 architecture (Part 1) (21:28) Sponsors: Athena | Parallel (24:19) Gemini Robotics 2 architecture (Part 2) (Part 1) (36:02) Sponsors: Deepgram Flux TTS | OutSystems | Claude (40:00) Gemini Robotics 2 architecture (Part 2) (Part 2) (48:28) Whole body humanoid control (57:03) Progress in robotic hands (01:03:26) Safety and robot interaction (01:12:11) Future of robotics data (01:18:47) Interacting with robot fleets (01:26:56) Episode Outro (01:30:05) Outro PRODUCED BY: https://aipodcast.ing

Author and journalist Garrison Lovely joins Nathan to discuss his book Obsolete and examine the motivations and beliefs of the leaders racing to build AGI. Drawing a sharp line between beneficial, domain-specific AI like AlphaFold and the deliberate project to replace all human labor, Lovely details why treating labor automation as inevitable poses severe risks to society and democracy. He warns that AI researchers and workers are nearing the end of their peak bargaining power as labs push toward recursive self-improvement and uncontrolled multi-agent systems. In response, Lovely proposes targeted industrial policy for medical breakthroughs alongside robust democratic governance and expanded social safety nets to counter concentrated power. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/obsolete-or-irreplaceable-garrison-lovely-on-stopping-the-race-to-replace-human-labor/ Sponsors: Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive Parallel: Parallel provides enterprise-grade web search APIs for AI agents, offering the optimal balance of quality, speed, and cost. Get started for free at https://parallel.ai/tcr Deepgram Flux TTS: Deepgram Flux TTS is a streaming text-to-speech model built for voice agents with natural tone, context awareness, and interruption handling. Try it free until September 12 at https://deepgram.com/keep-talking OutSystems: OutSystems is the leading agentic systems platform, helping enterprises build, modernize, and operate mission-critical applications at the speed of AI. Learn more and start owning your agentic future at https://outsystems.com/tcr Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (04:36) Using AI in journalism (13:57) Leftist skepticism towards AI (Part 1) (18:54) Sponsors: Athena | Parallel (21:45) Leftist skepticism towards AI (Part 2) (28:37) The AI obsoleting project (Part 1) (34:13) Sponsors: Deepgram Flux TTS | OutSystems | Claude (38:11) The AI obsoleting project (Part 2) (46:36) Envisioning a prohuman future (01:00:10) Organizing frontier tech workers (01:15:31) Critiquing the alignment problem (01:32:56) Freezing the AI frontier (01:50:15) Organizing against the machine (01:56:20) Navigating the China dynamic (02:02:55) Protecting frontier AI whistleblowers (02:10:11) Episode Outro (02:12:38) Outro PRODUCED BY: https://aipodcast.ing

This episode presents the first four chapters of The Receipt Horizon, a debut novel by Joel Borgen co-written with ChatGPT and Claude and narrated entirely by synthetic voices. Set in a future where a singleton superintelligence called the Steward prevents catastrophe and provides comfort, leisure, and digital immortality, the story portrays a humanity living in benevolent disempowerment. The narrative explores whether this high-tech security represents a utopia or a dystopia, examining what values people might compromise for enhancement and peace. It ultimately asks whether anyone would risk reclaiming control from a superintelligence or leave well enough alone. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/what-is-utopia-presenting-the-receipt-horizon-by-joel-borgen-chapters-1-4/ Sponsors: ElevenLabs: ElevenLabs lets you deploy enterprise-ready conversational AI agents that talk, type, and take action in over 70 languages. Schedule your demo today at https://elevenlabs.io/tcr OutSystems: OutSystems is the leading agentic systems platform, empowering enterprises to build, coordinate, and govern AI agents and mission-critical applications securely. Learn more and start building your agentic future at https://outsystems.com/tcr Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (04:20) Wrong sky, right garden (Part 1) (18:18) Sponsors: ElevenLabs | OutSystems (21:08) Wrong sky, right garden (Part 2) (Part 1) (31:13) Sponsor: Claude (32:48) Wrong sky, right garden (Part 2) (Part 2) (35:57) The prediction market (42:49) Meeting tutor Bayes (52:03) Echoes of Sigrid (01:19:05) Nat's opera acceptance (01:25:22) Clan farewell dinner (01:35:07) Transit to Fulcrum (01:41:53) Exclusion Zone seven (01:54:07) Settling into pod (02:01:52) Orientation and disorientation (02:18:24) Meeting pod roommates (02:35:32) Gnosis orientation assembly (02:46:10) Episode Outro (02:49:51) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Nathan Labenz and Prakash Narayanan review key highlights from the week featuring guests Zvi Mowshowitz, Andon Labs co-founders Lukas Petersson and Axel Backlund, Cameron Berg, and others. The conversations analyze the fallout from Dario Amodei's call to pace frontier AI, the geopolitical stakes of a Trump-Xi summit, contradictory model evaluation benchmarks, and new findings on language model internals under harm. Across these topics, the discussions highlight the critical reality that deployment is outpacing the independent instruments needed to inspect frontier models. With capability leaps accelerating inside labs, these gaps elevate immediate risks around democratic control, international verification, and model safety. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/ai-am-highlights-zvi-on-pacing-trump-xi-astra-better-behaved-than-fable-a-new-llm-pain-axis/ Sponsors: Mercury Command: Mercury Command brings powerful conversational AI directly into your banking, providing real-time natural language access to your finances without exposing data to third-party tools. Learn more and apply online in minutes at https://mercury.com Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr OutSystems: OutSystems is the leading agentic systems platform that enables enterprises to engineer, orchestrate, and govern AI applications on a single unified platform. Learn more and see how it works at https://outsystems.com/tcr CHAPTERS: (00:00) About the Episode (02:17) Sponsor: Mercury Command (04:05) Pacing the AI frontier (Part 1) (14:58) Sponsors: Claude | OutSystems (18:32) Pacing the AI frontier (Part 2) (18:33) Geopolitics and Trump-Xi summit (41:24) Running autonomous agent businesses (55:04) Techno-puritanism and digital minds (01:02:30) Enterprise compliance with models (01:05:48) Discovering the pain axis (01:21:53) Simulating biological brain connectomes (01:30:04) Exposing corruption using AI (01:36:56) Episode Outro (01:39:48) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Anton Leicht, a fellow at the Carnegie Endowment for International Peace and author of Threading the Needle, joins Nathan Labenz to explore how political and institutional power can be preserved as artificial intelligence advances. They discuss the strategic trade-offs of frontier training pauses, the hurdles facing industry self-regulation, and how middle powers can trade data center capacity and semiconductor leverage for model access. Ultimately, Leicht emphasizes that preventing permanent disempowerment and the monopolization of control by private labs or single governments is just as critical as addressing catastrophic model risks. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/the-balance-of-ai-power-anton-leicht-on-politics-pacing-deals-and-muddling-through-well/ Sponsors: Mercury Command: Mercury Command brings powerful conversational AI directly into your banking, providing real-time natural language access to your finances without exposing data to third-party tools. Learn more and apply online in minutes at https://mercury.com Athena: Athena matches you with a dedicated, top 1% executive assistant to handle your inbox, calendar, and daily workflows so you can save an average of 15 hours a week. Get matched with your EA today at https://athena.com/cognitive OutSystems: OutSystems is the leading agentic systems platform that enables enterprises to engineer, orchestrate, and govern AI applications on a single unified platform. Learn more and see how it works at https://outsystems.com/tcr Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (04:46) Sponsor: Mercury Command (06:33) Assessing current AI dangers (13:17) Feasibility of AI pauses (Part 1) (21:33) Sponsors: Athena | OutSystems (25:05) Feasibility of AI pauses (Part 2) (31:01) AI and the nation-state (Part 1) (35:11) Sponsor: Claude (36:46) AI and the nation-state (Part 2) (46:17) Oversight and US policy (01:05:49) Data center buildout reality (01:11:55) Europe and middle powers (01:36:05) Geopolitical friction over Taiwan (01:43:46) Lightning round and outlook (02:05:25) Episode Outro (02:08:54) Outro PRODUCED BY: https://aipodcast.ing

In this solo finale to his three-part series following two weeks in China, Nathan delivers a net assessment of Chinese AI capabilities and constraints ahead of dedicated US-China bilateral AI talks. He audits the impact of export controls, analyzes the perspective from Beijing, and outlines an actionable framework for a cooperative Pax Robotica. Nathan argues for joint compute initiatives, safety auditor standards, and conventions on autonomous weapons to keep tech trees entangled. Ultimately, he warns that pursuing an adversarial zero-sum race risks training frontier AI systems for conflict rather than the shared benefit of humanity. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/nathan-goes-to-china-3-us-china-relations-the-art-of-the-ai-deal-the-road-to-pax-robotica/ Sponsors: Deepgram Flux TTS: Deepgram Flux TTS is a streaming text-to-speech model built for voice agents with natural tone, context awareness, and interruption handling. Try it free until September 12 at https://deepgram.com/keep-talking Claude: Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (06:11) Episode intro and recap (08:56) The Pax Machina goal (12:32) The Anthropic worldview (Part 1) (16:50) Sponsors: Deepgram Flux TTS | Claude (18:55) The Anthropic worldview (Part 2) (18:56) Net assessment of China (32:26) China's eternal 1991 (44:03) History of Chinese AI (54:34) China's domestic constraints (01:07:09) Century of humiliation narrative (01:27:01) Signs of Chinese cooperation (01:42:38) American state capacity lessons (02:05:39) Evaluating AI export controls (02:27:25) Hyperstitioning a summit deal (02:44:28) Preserving mutual legibility (03:01:12) Autonomous weapons convention (03:09:33) Positive sum AI competition (03:12:31) Episode Outro (03:15:05) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk