“Is ProgramBench Impossible?” by frmsaul

About this episode

The episode discusses the challenges and implications of the ProgramBench coding benchmark that frontier models struggle to pass.

ProgramBench is a new coding benchmark that all frontier models fail spectacularly. We’ve been on a quest for “hard benchmarks” for a while so it's refreshing to see a benchmark where top models do badly. Unfortunately, ProgramBench has one big problem: it's impossible! What is ProgramBench? ProgramBench tests if a model can recreate a program from a “clean room” environment. The model is given only a bit of documentation and black-box access to the program (all the programs are CLIs), then tasked with re-implementing it. How does ProgramBench know if the implementation is correct? It also generates a bunch of unit tests for the program[1]. The re-implementing coding agent doesn't have access to any of those tests. The coding agent only considers a task “resolved” if it passes all of the tests and “almost resolved” if it passes 95% of them. Why is this problematic? Obscure behavior can enter the unit tests without being in the clean room path. An extreme version of this is a backdoor: program that behaves in one way most of the time but behaves totally differently when exposed to a specific string. This wouldn't make a task literally impossible, just incredibly hard in [...]…

People in this episode

Host: frmsaul

Topics covered

• coding benchmarks
• artificial intelligence
• programming
• model evaluation
• unit testing

Keywords

• ProgramBench
• coding benchmark
• AI models
• unit tests
• program re-implementation

More episodes of LessWrong (30+ Karma)

• “The Uncertainty That Matters Isn’t Fundamental” by jimmy · June 13, 2026 · 22 min
• [Linkpost] “US government directive to suspend access to Fable 5 and Mythos 5” by Capybasilisk · June 13, 2026 · 0 min
• “Claude Fable 5 and Mythos 5: The System Card” by Zvi · June 12, 2026 · 60 min
• “Citations Needed: Magic Encyclopedias to Save the World” by Oliver Sourbut · June 12, 2026 · 10 min
• “Simulating Simulators” by kromem · June 12, 2026 · 27 min
• “Implications of Continual Learning for LLM Agents: Introduction” by RohanS, Rauno Arike, Owen Terry, Achu Menon, Zhijing Jin, Francis Rhys Ward, Seth Herd · June 12, 2026 · 14 min