🔬Causal Models Need Causal Data - Xaira’s X-Cell model for Drug Discovery (Bo Wang & Ci Chu, Chief Discovery Officer & Chief AI Scientist)
About this episode
The episode discusses the importance of information-rich data in AI-driven drug development, featuring insights from Bo Wang and Ci Chu of Xaira Therapeutics.
Bet on information If test loss flatlines after 1.5B parameters while training loss continues to drop as you scale, that tells you that your model is limited by the amount of information in your data. Training on a single, smallish data set exposed an information gap: the 3.1B model falls off the scaling trend. Neither parameters nor compute will improve performance past this wall. For predicting changes to gene expression, you need more information rich data . This is what Chu and Bo’s teams have done, and here is what ~30x the information buys you: Now we can scale with parameters and training compute! We don’t know how much this effort costed, but we can guess that data collection experiments and infrastructure was a few tens of millions, and compute + headcount + research was a few million. The budget looks like a RL rollout budget, rather than a data rich pre-training one. We were lucky enough to have the two central figures in this story on our podcast. Taking the lead from Ci Chu and Bo Wang, Xaira Therapeutics is betting that information rich data is the key to AI-driven drug development. Chu was recently promoted to Chief Discovery Officer and Bo to Chief AI Scientist…
More episodes of Latent Space: The AI Engineer Podcast
- 🔬 The Lab of the Future Should Feel Like a Data Center — Andy Beam & Rafa Gómez-Bombarelli, Lila Sciences
- Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO
- 🔬 The Coolest Diffusion Research Isn't in LLMs — Evan Feinberg & Sergey Edunov, Genesis Molecular AI
- Why the Frontier Ecosystem must be Open — Matei Zaharia and Reynold Xin, Databricks
- Red-Teaming after Mythos — Zico Kolter & Matt Fredrikson, Gray Swan
- The Professor of Outputmaxxing — Anjney Midha, AMP
Explore listener stats, chart rankings, contacts and more on the Latent Space: The AI Engineer Podcast podcast page.