About this episode

Raymond Douglas discusses the complexities and potential pitfalls of AI for epistemics and coordination amidst increasing funding in the field.

Money is pouring in, people are looking for new areas to fund, and the invisible hand is starting to grab a bit at AI for epistemics and coordination (AIFEC hereafter). It also got invoked in AI2040 as a potential part of the wining strategy. My feelings here are mixed — I think the best version of AIFEC is great, but also the existing public writeups are only a few cycles deep on tracing out the different ways that the obvious plan backfires. And regrettably I think some people have correctly written off AIFEC because what they have read appears a bit naive to them. In my heart I always planned to do a proper writeup of my thoughts when things were a bit less busy, but, well, now we’re in the 100x funding era. So here's my scrappy, hopefully-better-than-nothing attempt. The six big claims: AIFEC could be great! It's a tractable way to do good on the margin, and the best version is a legitimate theory of victory It could also easily backfire, especially for implementations that depend on scaling with inference “Better epistemics” is often more hostile than it seems, and people have good reasons to be [...] --- Outline: (01:47) The basic case is pretty good! (04:04) Problem 1…

People in this episode

Guest: Raymond Douglas

Topics covered

• epistemics
• coordination
• AI
• funding
• strategy
• backfire
• tradeoffs

Keywords

• AIFEC
• funding
• epistemics
• coordination
• backfire
• strategy
• tradeoffs

Mentioned in this episode

Organizations: AI2040

More episodes of LessWrong (30+ Karma)

• “What do I mean by “Artificial General Intelligence”?” by Steven Byrnes · July 21, 2026 · 7 min
• “Differential acceleration of alignment-relevant capabilities is a bad bet” by Zephaniah Roe · July 21, 2026 · 16 min
• “Measuring Reward-Seeking via Contrastive Belief Updates” by Jérémy Scheurer, Axel Højmark, jenny, Felix Hofstätter, Theodore Ehrenborg, Bronson Schoen, Alex Meinke · July 21, 2026 · 29 min
• “11 Open Empirical Problems in Reward-Seeking” by Alex Meinke, Jérémy Scheurer, Axel Højmark, Theodore Ehrenborg · July 21, 2026 · 17 min
• “Demis Hassabis on the New Coming Age” by Zvi · July 21, 2026 · 21 min
• “AI 2040: Is it Actually a Deal?” by 1a3orn · July 21, 2026 · 16 min

Explore listener stats, chart rankings, contacts and more on the LessWrong (30+ Karma) podcast page.