“Blocking live failures with synchronous monitors” by James Lucassen, Adam Kaufman

About this episode

The episode discusses the importance of monitoring in AI control schemes, focusing on the differences between detection and prevention goals and the implications of monitor latency.

A common element in many AI control schemes is monitoring – using some model to review actions taken by an untrusted model in order to catch dangerous actions if they occur. Monitoring can serve two different goals. The first is detection: identifying misbehavior so you can understand it and prevent similar actions from happening in the future. For instance, some monitoring at AI companies is already used to identify behaviors like complying with misuse requests, reward hacking, and hallucination. The second goal is prevention: identifying misbehavior in order to mitigate the harmful effects of the particular action. For instance, constitutional classifiers are used by Anthropic to detect and prevent jailbreaks, and Claude Code auto mode prevents destructive mistakes in real time. The latency requirements for monitoring are different depending on whether the goal is detection or prevention. If the goal is detection, it's fine for there to be substantial latency in the monitor. But if the goal is prevention, then the monitor needs to trigger quickly, so that damage can be prevented or recovered from. In the extreme, monitors can run synchronously – after an action is generated by…

Topics covered

• AI control schemes
• monitoring
• detection
• prevention
• synchronous monitors

Keywords

• AI
• monitoring latency
• constitutional classifiers
• LLM

Mentioned in this episode

Products: Anthropic, Claude Code

More episodes of Redwood Research Blog

• “A taxonomy of barriers to trading with early misaligned AIs” by Alexa Pan · April 21, 2026 · 2h 6m
• “Introducing LinuxArena” by Tyler · April 20, 2026 · 9 min
• “Current AIs seem pretty misaligned to me” by Ryan Greenblatt · April 15, 2026 · 1h 5m
• “Anthropic repeatedly accidentally trained against the CoT, demonstrating inadequate processes” by Alex Mallen, Ryan Greenblatt · April 14, 2026 · 12 min
• “Logit ROCs: Monitor TPR is linear in FPR in logit space” by Kerrick Staley, Aryan Bhatt, Julian Stastny · April 12, 2026 · 28 min
• “If Mythos actually made Anthropic employees 4x more productive, I would radically shorten my timelines” by Ryan Greenblatt · April 11, 2026 · 13 min