Running data-driven evaluations of AI engineering tools

About this episode

This episode discusses a practical model for evaluating AI engineering tools using data.

AI engineering tools are evolving fast. New coding assistants, debugging agents, and automation platforms emerge every month. Engineering leaders want to take advantage of these innovations while avoiding costly experiments that create more distraction than impact. In this episode of the Engineering Enablement podcast, host Laura Tacho and Abi Noda outline a practical model for evaluating AI tools with data. They explain how to shortlist tools by use case, run trials that mirror real development work, select representative cohorts, and ensure consistent support and enablement. They also highlight why baselines and frameworks like DX’s Core 4 and the AI Measurement Framework are essential for measuring impact. Where to find Laura Tacho: • LinkedIn: https://www.linkedin.com/in/lauratacho/ • X: https://x.com/rhein_wein • Website: https://lauratacho.com/ • Laura’s course (Measuring Engineering Performance and AI Impact): https://lauratacho.com/developer-productivity-metrics-course Where to find Abi Noda: • LinkedIn: https://www.linkedin.com/in/abinoda • Substack: ​​ https://substack.com/@abinoda In this episode, we cover: (00:00) Intro: Running a data-driven evaluation of AI tools…

People in this episode

Host: Laura Tacho

Guest: Abi Noda

Topics covered

• AI engineering tools
• data-driven evaluation
• tool selection
• engineering leadership
• impact measurement

Keywords

• AI tools
• evaluation model
• engineering leaders
• tool trials
• impact measurement

Mentioned in this episode

Organizations: DX

More episodes of Engineering Enablement by DX

• Designing the AI‑native engineering organization with 1Password, Microsoft and Atlassian · June 8, 2026 · 44 min
• Mapping the new SDLC at BNY: Codifying AI into every step of the delivery lifecycle (Jason Valentino) · June 8, 2026 · 33 min
• The current impact of AI on engineering velocity: What 400 companies are seeing (Abi Noda & Brian Houck) · June 8, 2026 · 27 min
• Beyond AI tools: Evolving software engineering organizations for the agentic era · June 8, 2026 · 30 min
• Assumptions as code: SiriusXM’s approach to platform prioritization · April 10, 2026 · 50 min
• Measuring AI impact, assessing readiness, and new data trends · April 3, 2026 · 38 min