Configuring a Versatile LLM Harness & Scraping the Web With Scrapy
About this episode
Ayan Pahwa discusses the importance of the harness around an LLM and efficient developer workflows for web scraping and self-hosting Python applications.
Which is more important, the model or the "harness" around an LLM? What are ways to assemble an efficient agentic developer workflow? This week on the show, Ayan Pahwa joins us to discuss harnessing, web scraping, and self-hosting Python applications.
People in this episode
Guest: Ayan Pahwa
Topics covered
- LLM
- web scraping
- Python applications
- developer workflow
- self-hosting
Keywords
- LLM
- harness
- web scraping
- Scrapy
- Python
- developer workflow
- self-hosting
More episodes of The Real Python Podcast
- Performance Engineering: Profiling and Making Apps Fast by Default
- Exploring Complex Systems & Maintainable Data Science Pipelines
- Navigating Silent Failures in AI: Strategies for Effective Oversight
- Improving NumPy Performance on Free-Threaded Python
- Programmatically Developing LLM Prompts With DSPy
- Should You Understand Your Entire Python Codebase?
Explore listener stats, chart rankings, contacts and more on the The Real Python Podcast podcast page.