How to Build Better AI Evals with Claude Code in 5 Steps | Shreya & Hamel
About this episode
Shreya and Hamel discuss AI evaluations and provide insights on building better AI evals.
📌 Copy the AI system I use to run a one-person $1M+ business: https://www.behindthecraft.com 🤝 Subscribe to my practical AI newsletter read by 150K+ professionals: https://creatoreconomy.so Shreya and Hamel teach the industry standard course for AI evaluations taken by 4,500+ students from OpenAI, Google, and more. In our interview, I asked them to give live feedback on the AI evals that I built for my skills and demo how anyone can run evals with ChatGPT/Claude and their free evals skill. Don’t miss this up to date primer on evals from the real experts. 🙂 P.S., I asked Shreya and Hamel to give my audience 25% off the next cohort of their AI evals course. It has 4.7 stars with 900 reviews which is almost unheard of on Maven. Sign up via the link here: https://maven.com/parlance-labs/evals?utm_campaign=peteryang&utm_medium=affiliate&utm_source=maven&promoCode=PETERYANG We talked about: (00:00) Why AI evals are completely different now with the latest models (02:30) Demo: Live audit of the evals that I built for my AI skills (05:14) Top down vs. bottom up evals and why AI sucks at the latter (08:16) Spinning up AI agents to grade each eval criteria (14:32) Demo…
People in this episode
Host: Peter Yang
Guests: Shreya, Hamel
Topics covered
- AI evaluations
- AI skills
- ChatGPT
- Claude
- live feedback
- automated evaluations
Keywords
- AI evaluations
- ChatGPT
- Claude
- live feedback
- automated evals
- AI skills
- eval criteria
- human judgement
Sponsors
Maven
Mentioned in this episode
Organizations: OpenAI, Google, Maven, Parlance Labs
Products: ChatGPT, Claude, AI evals course
More episodes of Behind the Craft
- AI Makes Cheating Easy. Here’s How It Can Make Kids Smarter Instead | Sue Khim
- How to Vibe Code Apps That Become Real Businesses | Amol Jain
- How I Run My 1.5M+ Follower Content Business With Codex | Riley Brown
- 5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway
- Hermes Co-Founder on Building an AI Agent That Improves Itself | Karan Malhotra
- ChatGPT Work + Codex Tutorial: My Complete System at OpenAI | Jason Liu
Explore listener stats, chart rankings, contacts and more on the Behind the Craft podcast page.