About this episode
This episode explores the challenges of applying AI in finance and discusses sophisticated methodologies to overcome them.
Why does every naive data scientist who tries to predict stock prices end up depressed? Finance systematically breaks standard AI. You'll discover the four methodological pitfalls: data scarcity (10 years of daily data = only 2,500 observations—laughably insufficient), look-ahead bias (accidentally using future data), the unconditional trap (models validate but can't predict what matters), and heavy tails (the rare crashes that define risk). The analogy that sticks: "It's like having an umbrella that doesn't work when it rains." But there's a solution. Task-driven training matches the P&L of benchmark strategies instead of learning impossible 10,000-dimensional distributions. You'll hear about dynamic portfolios that spontaneously switched hedging instruments during COVID, lasso regression for cost-effective hedging, and the "Persona Ledger" method—LLM-generated synthetic data with accounting rules as constraints. Finance breaks AI, but sophisticated methodologies are fixing it. Topics Covered - The "naive data scientist depression": why finance breaks standard AI - Four methodological pitfalls: data scarcity, look-ahead bias, unconditional trap, heavy tails - Task-driven…
People in this episode
Host: Basis Set
Topics covered
- Generative AI
- Finance
- Methodological pitfalls
- Task-driven training
- Dynamic portfolios
- Lasso regression
- AI challenges
Keywords
- data scarcity
- look-ahead bias
- heavy tails
- task-driven training
- dynamic portfolios
- lasso regression
- AI in finance
Mentioned in this episode
Organizations: COVID
Products: Persona Ledger
Explore listener stats, chart rankings, contacts and more on the NeurIPS 2025 by Basis Set podcast page.