When Nine Nines Isn't Enough

About this episode

The episode discusses the Oxide team's search for a source of hardware failures related to an ultra-reliable component.

Bryan and Adam were joined by members of the Oxide team to describe the multi-year search for a mysterious source of hardware failures. All related to an ultra-reliable--and yet still not reliable enough--component. In addition to Bryan Cantrill and Adam Leventhal , we were joined by members of the Oxide team: Nathanael Huffman , Alan Hanson, Cliff Biffle , Eric Aasen , Robert "RFK" Keith , and Jordan Hendricks . Some of the topics we hit on, in the order that we hit them: Oxide RFD 630: BMR491 Glitch Mitigation Plan RFK's report PRs needed! If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar . We'd love to have you join us, as we always love to hear from new speakers!

People in this episode

Hosts: Bryan Cantrill, Adam Leventhal

Guests: Nathanael Huffman, Alan Hanson, Cliff Biffle, Eric Aasen, Robert "RFK" Keith, Jordan Hendricks

Topics covered

• hardware failures
• ultra-reliable components
• glitch mitigation
• Oxide team
• technology discussions

Keywords

• hardware failures
• Oxide team
• glitch mitigation
• technology
• reliability

Mentioned in this episode

Organizations: Oxide Computer Company, Oxide

Books & works: Oxide RFD 630: BMR491 Glitch Mitigation Plan

More episodes of Oxide and Friends

• This Old Repo: LLMs and the Restoration of BattleTris · June 9, 2026 · 1h 18m
• Rooting for the Home Team with Paul Freedman and Bryan Carmel · May 27, 2026 · 1h 2m
• The Tale of Reverso · May 16, 2026 · 1h 7m
• AI in Computer Science Education · May 10, 2026 · 1h 29m
• Mechanical Engineering at Oxide [chapter images] · May 7, 2026 · 1h 25m
• Are LLMs Insufficently Lazy? · May 3, 2026 · 1h 31m