Doom Debates!

***ATTENTION: We’re looking for paid help at LessOnline from June 5-7. Apply now!***OpenAI pushes the frontier of mathematics, Anthropic hires Andrej Karpathy to work on recursive self-improvement, live callers challenge the doom argument, and we've got a soundboard!Doom Debates Paid Gig in Berkeley!Doom Debates is proudly sponsoring The LessOnline conference this year and we got a merch table in the courtyard! We need 2 people to help represent the show — hand out T-shirts, talk to attendees, and spread the good word about Doom Debates. If you’re a regular viewer of the show, you probably know enough. Just be knowledgeable, friendly and helpful!* 💰We pay you $500 to help out during the entire Fri-Sun event* 🎟️ Plus you get FREE admission to the conference ($675 value)This is best suited for someone local to the SF Bay Area or who can arrange accommodations. If you’re interested, send an email to internship@doomdebates.com or DM Producer Ori on Discord. Tell us a little about yourself and your availability for all 3 days. Thanks!LinksLess Online conference (June 5–7, Berkeley) — https://less.onlineDoom Debates Discord — https://discord.gg/g2X8h5UCrUOpenAI — “An OpenAI model has disproved a central conjecture in discrete geometry” — https://openai.com/index/model-disproves-discrete-geometry-conjecture/Timestamps00:00:00 — Welcome + New Soundboard!00:09:27 — Doom Debates Is Sponsoring Less Online00:17:30 — Ben Goertzel Episode Recap00:27:30 — Goertzel's Alignment Blind Spot00:34:21 — OpenAI Disproves an 80-Year-Old Erdős Conjecture00:41:00 — David Deutsch Says "Amazing"00:50:31 — Eliezer Yudkowsky's Job Replacement Progression00:54:07 — Anthropic's $44 Billion Run Rate01:07:45 — Schadenfreude Week: Liron Loses $10K01:13:04 — Teaching Ezra Prediction Markets01:18:11 — Caller Williawa: Is the Goal Engine Argument Effective?01:37:00 — Caller Will: Brain Rot and Idiocracy01:55:04 — Anthropic Hires Karpathy for Recursive Self-Improvement01:57:16 — Rob Wiblin Says Recursive Self-Improvement Should be "Illegal"02:01:37 — Caller Henry: The Chicken Riddle02:08:41 — David Sacks Kills the AI Executive Order02:11:20 — The Philanthropy Tidal Wave02:15:56 — Donate to Doom DebatesDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Ben Goertzel is a pioneering AI researcher, entrepreneur, and author who has helped organize AGI as a research field since the 2000s through books, workshops, and conferences.We talk about the early days of AGI research with Yudkowsky, get Ben’s timeline to the singularity, and I try to understand what makes him so optimistic that superintelligence will be beneficial for humanity.Timestamps00:00:00 — Cold Open00:00:59 — Introducing Dr. Ben Goertzel00:04:11 — How Did Ben Get Interested in AGI?00:08:26 — Early AGI Enthusiast Community: Extropians & SL400:15:15 — Is There Headroom Above Human Intelligence?00:19:53 — Can Superintelligent AI Kill Everyone?00:28:25 — Ben’s AGI Timeline: 202900:36:00 — What Modern AI Is Missing00:42:01 — AGI to Superintelligence — FOOM in a Matter of Months00:46:34 — What’s Your P(Doom)?™00:53:37 — Ben’s Case for AI Acceleration01:03:50 — The Orthogonality Thesis Is “Dishonest”01:13:43 — Liron: Moore’s Law for Goal Achievement01:23:50 — Ben: Alignment Is Just Good Parenting01:27:36 — Alignment in Practice: OmegaCLAW01:35:47 — Creating Stable Goals Under Self-Modification01:45:22 — SingularityNET, OpenBGI, and the ASI Alliance01:57:31 — The $ASI Token Economy02:01:15 — ClosingLinksBen Goertzel on X — https://x.com/bengoertzelSingularityNET — https://singularitynet.io/OpenBGI — https://openbgi.com/Hyperon AGI Framework — https://hyperon.dev/OmegaCLAW on GitHub — https://github.com/asi-alliance/OmegaClaw-CoreASI Alliance — https://superintelligence.io/The Consciousness Explosion (book) — https://theconsciousnessexplosion.ai/“The Singularity Institute’s Scary Idea (and Why I Don’t Buy It)” (2010) — http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.htmlKen Stanley Debate — https://lironshapira.substack.com/p/debate-with-a-former-openai-researchDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

In December 2023, I joined the Theo Jaffee Podcast to talk about a wide range of topics including non-AI x-risks, where I disagree with Eliezer Yudkwowsky, cryptocurrency and if P(Doom) is rigorous. This episode was recorded 6 months before I'd started Doom Debates. Let's look back with the power of hindsight and see if my answers still hold up.Links#10: Liron Shapira — AI doom, FOOM, rationalism, and crypto (YouTube) — https://youtu.be/YfEcAtHExFMWatch Theo Jafee now on MTS — https://www.mts.now/MTS Live — https://x.com/MTSliveTimestamps00:00:00 — Theo's Introduction00:01:03 — Is Liron Worried About Non-AI Existential Risks00:03:36 — Suffering Risks00:05:22 — Is P(Doom) rigorous?00:09:42 — Is Eliezer Overestimating P(Doom)?00:12:18 — Where Does Liron Disagree with Eliezer?00:15:43 — What Would Change Liron's Mind00:17:14 — Elon Musk's AI Timelines00:18:34 — Is xAI Making Things Worse?00:20:24 — The Case for an AI Manhattan Project00:22:32 — Are Superforecasters Wrong About AI Risk?00:26:02 — The Race Against Time for Alignment00:28:01 — Headroom Above Human Intelligence00:33:31 — Vitalik's d/acc Framework00:35:23 — Edging Toward Superintelligence00:38:21 — From Chatbot to World-Ender00:41:55 — GPT Paradigm vs. AlphaZero00:43:18 — Critiquing AI Optimism00:48:29 — Deceptive Alignment and Gradient Descent00:53:31 — Does Nice Training Data Make Nice AI?00:57:57 — How Do You Live with 50% P(Doom)?01:00:08 — Why Have Kids If the World Might End?01:02:16 — Israel vs Hamas01:06:15 — How LessWrong Changed Liron's Life01:08:42 — Rationalism and Effective Altruism01:14:49 — Why Blockchain Has No Use Case Beyond Cryptocurrency01:22:13 — Charlie Munger and Richard Feynman01:24:38 — ClosingDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Claire Berlinski is a journalist, Oxford PhD, and author of The Cosmopolitan Globalist.She invited me to her weekly symposium to make the case for AI as an existential risk. Can we convince her sharp, skeptical audience that P(Doom) is high?Subscribe to The Cosmopolitan Globalist: https://claireberlinski.substack.com/Follow Claire on X: https://x.com/ClaireBerlinski“If Anyone Builds It, Everyone Dies” by Eliezer Yudkowsky & Nate Soares — https://ifanyonebuildsit.comTimestamps00:00:00 — Introduction00:02:10 — Welcome and Setting the Stage00:06:16 — Outcome Steering: The Magic of Intelligence00:10:40 — Collective Intelligence and the Path to ASI00:12:53 — The Five-Point Argument00:14:56 — The Alignment Problem and Control00:17:56 — The Genie Problem and Recursive Self-Improvement00:20:38 — Timeline: Five Years or Fifty?00:26:14 — Social Revolution and Pausing AI00:28:54 — Energy Constraints and Resource Limits00:31:23 — Morality, Empathy, and Superintelligence00:37:45 — How AI Is Actually Built00:38:31 — Computational Irreducibility and Co-Evolution00:44:57 — Foom and the Discontinuity Question00:46:44 — US-China Rivalry and the Arms Race00:49:36 — The Co-Evolution Argument00:55:36 — Alignment as Psychoanalysis00:57:24 — Anthropic’s “Harmless Slop” Paper01:00:33 — Policy Solutions: The Pause Button01:04:47 — Military AI and the Singularity01:07:10 — Cognitive Obstacles and Doom Fatigue01:09:07 — Why People Don’t Act01:13:00 — Reaching Representatives and Building a Platform01:17:12 — Sam Altman and the Manhattan Project Parallel01:19:14 — Community Building and Pause AI01:22:07 — Call to Action and ClosingDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Multiple live callers join this month's Q&A as we cover the imminent demise of programming as a profession, the Anthropic/Pentagon showdown, and debate the finer details of wireheading. I clarify my recent AI doom belief updates, and then the man behind Roko's Basilisk crashes the stream to argue I haven't updated nearly far enough.Timestamps00:00:00 — Cold Open00:00:56 — Welcome to the Livestream & Taking Questions from Chat00:12:44 — Anonymous Caller Asks If Rationalists Should Prioritize Attention-Grabbing Protests00:18:30 — The Good Case Scenario00:26:00 — Hugh Chungus Joins the Stream00:30:54 — Producer Ori, Liron's Recent Alignment Updates00:43:47 — We're In an Era of Centaurs00:47:40 — Noah Smith's Updates on AGI and Alignment00:48:44 — Co Co Chats Cybersecurity00:57:32 — The Attacker's Advantage in Offense/Defense Balance01:02:55 — Anthropic vs The Pentagon01:06:20 — "We're Getting Frog Boiled"01:11:06 — Stoner AI & Debating the Finer Points of Wireheading01:25:00 — A Caller Backs the Penrose Argument01:34:01 — Greyson Dials In01:40:21 — Surprise Guest Joins & Says Alignment Isn't a Problem02:05:15 — More Q&A with Chat02:14:26 — Closing ThoughtsLinks* Liron on X — https://x.com/liron* AI 2027 — https://ai-2027.com/* “Good Luck, Have Fun, Don’t Die” (film) — https://www.imdb.com/title/tt38301748/* “The AI Doc” (film) — https://www.focusfeatures.com/the-ai-doc-or-how-i-became-an-apocaloptimistDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe