Smooth Scaling: System Design for High Traffic

Sam Dhar has spent 14 years building infrastructure at Cisco, Amazon Alexa, and Adobe, and now works as Senior Staff Engineer and AI infrastructure leader at Galileo, the enterprise AI evaluation platform. In this episode of the Smooth Scaling Podcast, Sam walks host Jose Quaresma through Cloud Prem: deploying your full product stack inside the customer's own cloud environment instead of running it as SaaS. They get into why the model is resurging, and it mostly comes down to data. Enterprises want ownership and control, plus a heavy compliance load (SOC 2, HIPAA, fully air-gapped government workloads), and they do not want a vendor sitting in the read path of their most sensitive data. Sam is candid about the hard parts. Cloud Prem can be a losing game on margins, deployment is the slowest thing in the pipeline, and every customer environment is different enough to reset the work. The conversation closes on AI: why it makes Cloud Prem urgent, the brutal GPU shortage, and why self-hosting an Opus-class model is still out of reach for most companies. A direct, practitioner-level look at where enterprise AI infrastructure is actually heading.Episode page---(00:00) - Intro (01:08) - What Cloud Prem actually is (06:05) - Why Cloud Prem is resurging now (09:37) - Provider, vendor, customer: who owns what (11:10) - "Data is paramount": the compliance driver (14:29) - Shipping software into someone else's environment (19:57) - When Cloud Prem becomes a losing game (26:48) - Quality, and the control plane / data plane split (28:50) - Monitoring without seeing the customer's data (30:52) - Why Sam moved to AI evals (34:56) - Self-hosting LLMs and the GPU bottleneck (38:01) - Smaller runtimes, frontier-level intelligence (41:46) - Why AI makes Cloud Prem urgent (46:59) - Rapid fire: the one book to read (49:01) - "Business equals scalability" Satyam “Sam” Dhar is a senior Staff Engineer and AI infrastructure leader at Galileo, where he designs systems that support real-time LLM workflows at enterprise scale. Prior to Galileo, he spent over six years at Adobe, contributing to AI-powered product development, evaluation platforms, and large-scale data systems. Earlier in his career at Amazon, he worked on high-throughput distributed services supporting Alexa’s device orchestration. Based in San Francisco, Sam’s insights and commentary have been featured in Newsweek, CNET, InfoQ, The New Stack, The Deep View, and others. He is also a Senior Member of the Institute of Electrical and Electronics Engineers.🔗 Connect Sam Dhar: https://www.linkedin.com/in/satyamdhar/ Host José Quaresma: https://www.linkedin.com/in/jose-quaresma/This podcast is researched by Joseph Thwaites, produced by Perseu Mandillo, and brought to you by Queue-it, your virtual waiting room partner.© Queue-it, 2026

In this episode, José Quaresma sits down with two Queue-it engineers — Zaigham Sarfaraz, Engineering Manager, and Šimon Bučko, Senior Software Engineer — to talk autoscaling in production. They cover the fundamentals of horizontal and vertical scaling, why stateless architecture matters for scaling out, and what happens when the metrics you're scaling on don't match your actual bottleneck. The conversation gets real when Zaigham shares a war story of autoscaling failing during an iPhone launch — one million users in one second — and how that experience reshaped how the team thinks about pre-scaling for extreme traffic. Šimon challenges the temptation to rely on default configurations and explains why the days you most need autoscaling to work are exactly the days it might not.Episode page---(00:00) - Introduction (00:46) - What is autoscaling under the hood? (03:25) - Why scaling down matters too (03:53) - Horizontal vs. vertical scaling (05:43) - When vertical scaling is the better choice (07:56) - Stateful vs. stateless applications (10:42) - Solving state for horizontal scaling (12:14) - The role of load balancers (14:31) - Choosing the right scaling metrics (16:46) - Is serverless the silver bullet? (21:34) - The cost paradox of autoscaling (23:40) - iPhone launch: when the whole world wants to buy a product (25:56) - Why autoscaling isn't enough for non-linear traffic (30:37) - The fallacy of the rule of thumb (32:48) - Rapid fire questions Šimon Bučko is a Senior Software Engineer at Queue-it, working across full-stack development. He is an AWS Certified Solutions Architect Professional with strong experience in software architecture and bridging the gap between business needs and technical execution. Zaigham Sarfaraz is an Engineering Manager at Queue-it with over 15 years of experience across frontend, backend, infrastructure, and people leadership. He is an AWS Certified Cloud Practitioner and plays a key role in ensuring stable system operations while contributing to the continuous improvement of Queue-it's backend architecture. This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo. © Queue-it, 2026

In this episode of Smooth Scaling, José Quaresma speaks with Hans Skovgaard, Chief Technology and Product Officer at Queue-it, about a shift that is already underway and accelerating fast: the internet now carries more automated bot traffic than human traffic — and agentic AI is about to make that gap much wider.Hans explains why the old model of "bots versus humans" is fundamentally broken, and why the real question is no longer who is visiting your site, but what their intent is. The conversation covers why autoscaling can no longer protect against the extreme traffic bursts that AI agents will generate, how to make bot attacks economically unviable, and what a future of AI agents buying concert tickets on your behalf actually looks like in practice. Hans also unpacks the evolving landscape of digital identity — from payment certificates to the EU Digital Identity Wallet — and what it means to build systems that can tell a genuine buyer from a scalper running 100,000 simultaneous requests.Episode page---(00:00) - Introduction (01:19) - The Internet Just Changed — More Bots Than Humans Online (03:51) - The New Threat Isn't Bots vs. Humans. It's Intent. (06:06) - Why Autoscaling Can't Save You in the Agentic Age (09:00) - Making Attacks Expensive — The Economics of Bot Defence (11:02) - What Does the Future Actually Look Like? The AI Agent Buying Your Tickets (14:30) - The Next Generation of Challenges — Easy for Humans, Costly for Bots (18:53) - The Deeper Problem: Volatility Is Going Out of Control (20:24) - Can We Prove You're Human? Identity, Trust & the EU Wallet (25:45) - Rapid Fire (30:07) - Outro Hans J. Skovgaard is Chief Technology and Product Officer at Queue-it, the Copenhagen-founded SaaS company whose virtual waiting room technology helps the world's biggest brands manage traffic surges and prevent bot abuse during high-demand online events. With over two decades of experience leading engineering and product organisations in Nordic software companies, Hans has built a career at the intersection of deep technical expertise and strategic leadership. Before Queue-it, he served as CTPO at Penneo, a Nasdaq Copenhagen-listed RegTech company, and as CTO and VP of R&D at Capture One, where he led the company's spin-off from Phase One, launched its first SaaS product, and shipped Capture One for iPad. Earlier, he held engineering leadership roles at Milestone Systems and Microsoft. He holds an M.Sc. in Artificial Intelligence from the University of Edinburgh and an MBA from IMD, and has published research at AAAI, IEEE, and ACM.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo. © Queue-it, 2026

In this episode of Smooth Scaling, José Quaresma talks with Ryan Sherlock, Director of Engineering at Intercom, about the realities of scaling databases in a fast-growing SaaS product. Ryan shares Intercom’s journey from a single MySQL database through Aurora, proxies, and per-customer scaling patterns—and what eventually pushed the team toward PlanetScale. The conversation also explores Intercom’s heartbeat-based approach to incident detection and response, focusing on customer impact rather than infrastructure metrics.Episode page---(00:00) - Intro and episode overview (01:14) - Early scaling pains: systems going down every day (02:56) - Database evolution: MySQL, caching, Aurora, and ProxySQL (07:36) - Tens of billions of rows and the table Intercom couldn’t migrate (09:07) - Intercom’s multi-region architecture and the EU region (10:59) - Why Intercom moved from Aurora to PlanetScale (Vitess) (15:12) - PlanetScale in practice: shards, VTGate, and zero-downtime upgrades (22:39) - Heartbeat metrics and automated incident response (30:03) - AWS outage case study: DynamoDB failure and real-time recovery (34:17) - Incident mitigation lessons: “I’m now a web box” and VTGate limits (41:40) - Rapid fire questions: books, career advice, and scalability mindset Ryan Sherlock is Senior Director of Engineering at Intercom in Dublin, where he leads the core technologies and infrastructure groups that power Intercom’s AI first customer service platform. Through talks and writing on the Intercom engineering blog, he shares practical playbooks on scaling infrastructure and engineering enablement, running high leverage incident response, and using heartbeat metrics to tie reliability directly to real customer outcomes rather than just server graphs. Outside Intercom, he serves on the board of the Rails Foundation, helping steward the future of the Ruby on Rails ecosystem. Before moving into tech leadership, Ryan spent several years as a professional cyclist, an experience he wrote about in “Why you should have skin in the engineering game”, and that still shapes how he thinks about risk, ownership, and reliability in software. This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo. © Queue-it, 2026

In this episode, Moji Sarooghi, Distinguished Product Architect at Queue-it, breaks down the design principles and distributed systems behind Queue-it’s virtual waiting room. He explains how the team handles massive traffic spikes, upholds strict first-in, first-out fairness on request, and maintains reliability at a scale that would overwhelm most platforms. Moji also covers the shift from server-side integrations to Edge compute, how Safety Net protects against unexpected peaks, and why simplicity and failure-oriented design drive every architectural choice. A clear, technical exploration of scaling responsibly when millions depend on your system.Episode page---(00:00) - Intro (01:34) - Visitor Flow: How the Waiting Room Works (03:24) - Edge vs. Server-Side Connectors (06:10) - Why Edge Improves Simplicity & Security (07:12) - Preventing Queue Bypass Attempts (09:14) - Connector Types & Verification Logic (12:04) - Safety Net: Automatic Peak Protection (14:54) - Scheduled Waiting Rooms + Safety Net (17:19) - FIFO at Scale (18:57) - Estimating Wait Times at Scale (20:40) - Designing for Reliability & High Traffic (24:38) - How Outflow Is Calculated (29:07) - Queue-It Token & Visitor Verification (31:02) - Cookies & Secure Access (32:35) - Key AWS Services in the Architecture (34:57) - Future: Multi-Cloud, Edge, & Bring Your Own Proxy (37:59) - Outro Mojtaba Sarooghi is a Distinguished Product Architect at Queue-it. Moji was one of the company’s first employees, starting his journey as a software developer over 10 years ago. He is highly experienced with AWS services, product and architectural design, managing developer teams, and defining and executing on product vision.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo. © Queue-it, 2025

In this episode, Gabor Sivók, Cloud Security Architect at Queue-it, shares a practical look at what it takes to secure large-scale systems in today’s cloud environments. He walks through Queue-it’s journey to ISO 27001 certification, the real trade-offs between security and performance, and how security practices adapt in multi-cloud setups. Gabor also weighs in on the growing role of AI in security operations—and why the best security work often stays invisible. A grounded conversation for anyone working at the intersection of reliability, scalability, and security.Episode page---(00:00) - Intro & welcome to the Smooth Scaling Podcast (02:58) - AWS GameDay: Security through gamification (06:31) - The journey to ISO 27001 certification (09:34) - Balancing scalability, reliability & security (12:49) - What TLS really means for secure communication (15:29) - Moving from AWS to multi-cloud security (18:31) - How AI is changing cloud security (20:27) - The endless game of attackers vs. defenders (22:44) - Advice for starting a security career early (24:11) - Wrap-up & closing message Gabor Sivók is a Cloud Security Architect at Queue-it, where he leads security efforts across the R&D organization. With a background in infrastructure and compliance, he played a key role in Queue-it’s ISO 27001 certification and now focuses on securing multi-cloud environments at scale. Gabor works closely with platform engineering teams to embed security into architecture decisions while balancing performance, resilience, and risk. He’s also an active participant in the security community, keeping pace with emerging threats and tooling through Discord, Reddit, and bug bounty networks.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo. © Queue-it, 2025

In this episode of the Smooth Scaling Podcast, Ilia Bromberg (Akamai) and Martin Larsen (Queue-it) explore the evolution of bots, the growing complexity of detecting them, and the real-world impact on hype events like product drops and ticket sales. They introduce Hype Event Protection, a new joint solution from Queue-it and Akamai, designed to level the playing field for genuine users. The discussion covers technical approaches to bot mitigation, performance optimization, and the importance of layered defenses for high-demand online events.Episode page---(00:00) - Welcome & Guest Introductions (01:01) - Why Bots Are a Problem (04:06) - Good Bots vs. Bad Bots (07:49) - How Bots Have Evolved (11:42) - Bots Move Into E-commerce (13:10) - Residential IPs and Hidden Networks (15:42) - What Is a Hype Event? (18:46) - Why Queue-it and Akamai Partnered (22:20) - Fairness, Trust & Brand Reputation (28:36) - How Hype Event Protection Works (35:59) - Preparing for Big Events (44:34) - Real Results from Beta Customers (46:42) - How to Get Started & Wrap-Up Ilia Bromberg is a Principal Solutions Engineer at Akamai Technologies with nearly 30 years experience helping organizations secure and scale their digital environments. A seasoned leader in web and application security, he has been named Akamai’s Solutions Engineer of the Year and has earned multiple hackathon and innovation awards. He holds CISSP, CCSP, and GWAPT certifications and specializes in WAFs, bot management, API security, DNS, and zero trust technologies. Martin Larsen is a Distinguished Product Architect at Queue-it. Starting as a software developer, Martin was one of the company’s first employees. He played an instrumental role in building the foundations of Queue-it and is heavily involved in activities including the design, architecture, testing, and deployment of the virtual waiting room, as well as defining and executing on product vision. This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo. © Queue-it, 2025

In this episode of Smooth Scaling, Jillian Als, CMO at Queue-it, unpacks The Age of Online Trust report. She explores why reliability is the license to operate, how trust is earned in drops but lost in buckets, and what 1,000 consumers revealed about their expectations for fairness, transparency, and resilient digital experiences. For technical leaders, the findings confirm that every percentage point of uptime and performance directly impacts trust, loyalty, and long-term business growth.Episode page---Jillian Als is Chief Marketing Officer at Queue-it, where she leads global marketing efforts to help businesses earn and protect online trust for billions of digital visitors each year. With 15+ years in B2B SaaS marketing, she’s known for her expertise in go-to-market strategy, demand generation, and brand development, as well as her passion for building happy, high-performing teams. A frequent speaker at industry podcasts and events like SaaSiest2025 and Funnel Vision, Jillian brings a deep understanding of consumer behavior and the link between digital performance, transparency, and loyalty.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo.(00:00) - Intro: Trust, Scale & a Special Guest (01:06) - The Meaning of Reliability (03:08) - Exploring Technical and Commercial Views on Reliability (04:53) - A Deep Dive Into the “Age of Online Trust” Report (07:13) - The Global Survey Methodology (08:13) - The Definition of Online Trust (10:28) - The Ongoing Importance of Trust Beyond Peak Events (12:33) - Key Findings: How Bad Experiences Erode Trust (13:54) - Gen Z’s Higher Trust Expectations (16:26) - Preference for Smooth Experiences Over Speed (18:45) - The Psychology Behind Informed Waiting (21:11) - How Trust Fuels Loyalty, Spend, and Advocacy (26:31) - Technical Takeaways From the Report (29:06) - Rapid Fire Insights on Scalability, Books, and Career Advice © Queue-it, 2025

In this episode, Barnaby Clark, CEO of Line-Up, reveals the engineering practices behind resilient ticketing systems that handle real-world demand. Barnaby explains how Line-Up rebuilt their platform from the ground up to meet the complex needs of live events, from unique inventory structures and API scaling to predictive load handling and third-party integrations. Barnaby dives into the evolving threat of bots, the nuances of asynchronous payments, and how to design for bursts in traffic without breaking the customer experience. It’s a practical look at infrastructure, performance, and the unpredictable nature of ticketing at scale.Episode page---Barnaby Clark is CEO and Co-Founder at Line-Up. He has 12 years of experience designing innovative software products across diverse stacks, scaling and guiding cross-functional teams, building high-growth e-commerce platforms, and overcoming complex software challenges. Line-Up was shortlisted for Best Technology Provider at the British Media Awards, won Seedcamp London and has secured multiple funding rounds from angel investors, institutional backers, and corporate entities. Prior to Line-Up, Barnaby spent 5 years working on Mergers & Acquisitions and private capital fundraising efforts within the technology sector. This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo.(00:00) - Introduction: Designing Scalable Ticketing Systems (00:45) - Barnaby’s Journey to Founding Line-Up (02:15) - Unique Scaling Challenges in Ticket Sales (05:08) - Breaking Down the Ticket Purchase Journey (06:41) - Read vs. Write Operations in Scalability (09:53) - Handling Sudden Traffic Spikes (12:11) - Predictive Scaling and Early User Signals (17:26) - Integrating Third-Party Ticket Sales APIs (20:02) - Payment Providers and Asynchronous Challenges (26:09) - Disaster Recovery and System Protection (30:35) - Tackling Bots and Fraud in Ticketing (35:04) - Rapid-Fire Insights & Recommendations © Queue-it, 2025

In this episode, Kolton Andrus, Founder and CEO of Gremlin deep dives into all things chaos engineering and reliability testing. Kolton shares his journey from leading reliability efforts at Amazon and Netflix to founding Gremlin, an enterprise reliability platform. They discuss what it really takes to build resilient systems, the cultural shift required to prioritize reliability, and how Gremlin is working to reshape accountability in engineering teams. From testing dependencies to aligning incentives, this conversation is packed with real-world insights into scaling systems (and teams) that don't break under pressure.Episode page---Kolton Andrus is the CEO and founder of Gremlin. Prior, he focused on building and operating reliable systems at Netflix and Amazon. At both companies he operated systems at scale, managed company wide incidents and helped build out their respective reliability programs and toolsets.Host Jose Quaresma is the VP of Technical Engagement at Queue-it, working on the frontlines with some of the world’s biggest businesses on their busiest days, from Ticketmaster to Zalando to Home Office U.K. Each week, he’ll be joined by experts across industries, uncovering how major organizations design, build, and deploy systems that perform at scale.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo.(00:00) - Intro & Guest: Kolton Andrus (04:20) - Founding Gremlin (2016) (08:47) - Rewarding Invisible Reliability Work (12:27) - Proving Reliability’s Business Value (15:21) - Rethinking the “Chaos Engineering” Label (20:18) - Chaos Testing to Reliability Scores (24:25) - Spreading Reliability Culture Across Teams (28:50) - Safe, Incremental Failure Testing in Prod (33:30) - Load + Fault Testing for Peak Traffic (36:30) - AI’s Opportunities & Risks for Ops (39:30) - Defining Scalability as Elasticity (44:18) - Key Takeaways & Farewell © Queue-it, 2025

In this episode, seasoned platform engineer Tristan Watson shares his learnings from handling peak traffic at Rapha and Booking.com. Tristan reveals the key challenges, trade-offs, and best practices involved in preparing infrastructure for high-traffic product drops and collaborations. Whether you're navigating traffic surges or optimizing for resilience, Tristan’s advice will help you prepare your systems to handle the pressure.Episode page---This episode´s guest is Tristan Watson. Tristan Watson has spent over a decade mastering the art of keeping websites fast, stable, and scalable. With experience leading teams and steering key projects across tech, retail, and finance he consistently balances technical excellence with business goals. His pragmatic approach and passion for emerging tech like AI make him a sought-after consultant. Off the clock, you’ll find him exploring new tech trends or out on a bike ride. You can find Tristan on LinkedIn here.Host Jose Quaresma is the VP of Technical Engagement at Queue-it, working on the frontlines with some of the world’s biggest businesses on their busiest days, from Ticketmaster to Zalando to Home Office U.K. Each week, he’ll be joined by experts across industries, uncovering how major organizations design, build, and deploy systems that perform at scale.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo.(00:00) - Intro (00:58) - Tristan's journey (02:47) - Differences in scalability (08:08) - Differences in traffic peaks (11:12) - The challenges of an SRE team (16:34) - High stakes make the most memorable moments (19:39) - The Rapha system setup in more detail (26:24) - Iterating - anticipating problems or learning from mistakes (27:57) - The alternatives on the table (29:30) - Uncertainty in the reliability of the current systems (30:59) - The virtual waiting room (33:13) - Experience during the drop (37:03) - The best moments are with great partners (40:00) - Main learnings from Product drop (42:04) - Rapid Fire Questions (46:07) - Outro © Queue-it, 2025

In this episode, Yam Shal-Bar, CTO at Radview, discusses the the evolving world of load testing and how it's used to prepare for peak traffic. He covers the most common system bottlenecks, the importance of iterative testing, and strategies for accurately simulating user journeys. Yam shares insights into common misconceptions around testing, best practices, and trends like AI for test analysis and API-level testing. Whether you're launching a new web app or tuning an existing one, this episode is packed with practical advice for testing systems for resilience and scalability.Episode page---This episode's guest is Yam Shal-Bar. Yam Shal-Bar is an experienced development leader and software architect with over two decades of expertise in managing distributed teams and delivering enterprise-scale software solutions. As CTO at RadView Software, he drives the company’s technical roadmap and leads the development of core products, including RadView’s performance testing platform and web dashboard. Throughout his career—including leadership roles at British Telecom, Reliance Infocomm, and Vodafone—he has championed Agile methodologies, DevOps practices, and CI/CD pipelines to deliver robust, scalable systems.Host Jose Quaresma is the VP of Technical Engagement at Queue-it, working on the frontlines with some of the world’s biggest businesses on their busiest days, from Ticketmaster to Zalando to Home Office U.K. Each week, he’ll be joined by experts across industries, uncovering how major organizations design, build, and deploy systems that perform at scale.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo.(00:00) - Intro (00:59) - Yam Shal-Bar and Radview (02:20) - Simulating User Journeys (05:00) - How to approach load testing? (06:25) - What are common finds when load testing? (10:42) - Different perspectives on "capacity" (16:05) - The playbook of load testing (18:50) - What are common bottlenecks in complex systems? (20:32) - Third-party services and load testing (23:33) - What exactly are you load testing? (26:11) - What is changing within the load testing space? (27:39) - API and User Journey testing (30:56) - Rapid-Fire Questions (35:25) - Wrap Up © Queue-it, 2025

No system has 100% reliability. Failures and faults are inevitable. At scale, everything breaks. In this episode, Martin Larsen explains the design for failure approach behind Queue-it’s architecture and how it increases the platform’s availability and resilience. Larsen explores the principles behind designing for failure, the tradeoffs involved, the mechanisms implemented at Queue-it, and the tangible ways companies can bring this development approach into their processes.Episode page---Host Jose Quaresma is the VP of Technical Engagement at Queue-it, working on the frontlines with some of the world’s biggest businesses on their busiest days, from Ticketmaster to Zalando to Home Office U.K. Each week, he’ll be joined by experts across industries, uncovering how major organizations design, build, and deploy systems that perform at scale.This episode's guest is Martin Larsen, a Distinguished Product Architect at Queue-it. Starting as a software developer, Martin was one of the company’s first employees. He played an instrumental role in building the foundations of Queue-it and is heavily involved in activities including the design, architecture, testing, and deployment of the virtual waiting room, as well as defining and executing on product vision.This podcast is hosted by José Quaresma, researched by Joseph Thwaites and produced by Perseu Mandillo.© Queue-it, 2025