Agentic Conversations (formally mlops.community)

A large language model knows everything and nothing at the same time. Tim O’Reilly, founder and CEO of O’Reilly Media, joins us ahead of his keynote at AGNTCon + MCPCon North America to talk about what that paradox means for the people building with these models.We start with what Tim is bringing to the stage in San Jose, then move to the craft of steering a model. The right words work almost like spells that open up new regions of latent space. Something as small as telling a model to use a computer can unlock a new tier of capability, where agents go off and do real work.From there, the conversation turns philosophical. Tim makes the case for mechanism design: shaping the rules of a system so the outcomes we want become the natural ones. We connect it to a book, The Score, and the question of which game we are really playing. In Risk, everyone fights over the board. In charades, everyone plays together. We close on Tim’s view that AI should be an ecosystem rather than a product that sucks the air out of the room, so that everyone can build on top of it and flourish.Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmTim O’Reilly: https://www.linkedin.com/in/timo3Timestamps:[00:00] Cold open[01:02] Grappling with the bitter lesson[02:22] The value of human expertise[04:54] Why AI needs us[05:37] Culture is what remains[06:23] The toil audit story[08:26] AI knows everything and nothing[08:57] Magic words and runaway agents[11:18] Brute force vs human context[12:32] Turning scar tissue into infrastructure[14:12] Google and the first bitter lesson[16:25] Silicon Valley's sad turn[17:43] Artificial volition[19:08] Casting spells in latent space[20:46] The peaceful atom lesson[22:15] Sharing value with creators[24:54] No jobs means no customers[25:43] Mechanism design explained[26:35] Unix and the architecture of participation[28:45] The missing mechanisms of the AI economy[29:36] What game are you playing[30:43] The game of human flourishing[32:04] Big labs vs open source AI[33:47] Wrap up

What happens when you give 200 engineers who have barely touched AI a coding assistant and a six-month deadline to rebuild their platform? Pepijn Senders, a principal engineer who previously led AI enablement at HelloFresh, has done exactly that, and he chose Sonnet over Opus on purpose.His reasoning is what he calls "Sonnet maxxing." A model that reasons less and stops sooner keeps developers involved, and that matters a lot when the codebase is banking software full of hard business rules. The result was 800,000 lines of code, about 60% of them written by Sonnet, shipped by a team that never stopped thinking for itself. Pep also explains how to justify the spend once the token-maxxing era ends and the CFO starts asking questions. He covers a scatter chart that plots token spend against output, the uncomfortable question of whether a 1% better home screen was worth $20K in tokens, why he measures gains in headcount rather than 2x or 3x multipliers, and where he fell short when trying to prove ROI at HelloFresh.We also get into a codebase test you can run yourself, why AI recommendations quietly narrow your choices, and multiplayer refinement sessions where the whole team works through the agent's questions together. Pep shares an experiment where designers and engineers got the same one-shot prompt, and it showed how words like "scalable" and "maintainable" steer a model. We finish with how his team built a meta harness on top of Claude Code, after finding that 85% of their code followed the same path: refinement, specification, implementation, verification.Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmPepijn Senders: https://www.linkedin.com/in/pepijn-sendersTimestamps:[0:00] Can you still read your codebase[1:15] Meet Pep from HelloFresh[2:35] What is Sonnet Maxing[3:41] Rebuilding a platform in six months[5:02] Why Sonnet made adoption easier than Copilot[6:37] Proving the ROI of AI coding[8:50] Fighting for AI ROI at HelloFresh[11:02] Token spend vs output[13:24] Tinder pickup lines on the company dime[14:13] Measuring ROI in headcount saved[18:26] Keeping humans in control of AI code[22:26] When Opus and Fable stop speaking English[24:19] Multiplayer refinement sessions with AI[28:16] Magic words that steer the model[32:32] Inside the company Meta harnesses

What happens to the web when the ones browsing it are no longer people? Sarah Drasner, Area Tech Lead at Google, sits down with us ahead of her Agentic Web keynote at AGNTCon + MCPCon North America. She works through that question from both directions: rebuild the web so agents can use it properly, or build agents smart enough to handle the web as it already exists.We get into the split between computer-use and browser-use agents on one side and API and MCP agents on the other. We also watch a real example of the idea in action: an agent pulling speaker photos from Flickr, hunting down company logos, and loading everything into Canva so the human only touches the creative part. From there, Sarah shares the unlikely origin story of WebMCP (and who's to blame for it). We then pressure-test a bold idea: a WebMCP gateway that could cover the huge share of websites that will never ship support on their own.Along the way, we cover the instinct that separates senior engineers, the ability to sense which business problems are actually worth solving. We also talk about how thought experiments cut through a pile of competing priorities, the hacky WebMCP workarounds already showing up in the wild, and why the best agent experiences are the ones that delight customers in ways they never saw coming.Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmSarah Drasner: https://www.linkedin.com/in/sarahdrasnerTimestamps:[0:00] Cold open[0:52] The AGNTCon keynote[1:23] Change the web or fix agents[3:35] Browser agents vs MCP agents[5:48] An agent that builds thumbnails[7:03] Keeping the creative work human[9:12] Agent allowances and engineering instinct[13:34] Thought experiments and priorities[15:23] The WebMCP origin story[17:08] A WebMCP gateway for the web[18:53] It works, but it is not ideal[19:44] Hacky WebMCP in the wild

What happens to an MMORPG when half the players never log off? Issac Lee, who runs AI and blockchain experiments at NEXUS, has been testing that idea. His team is filling game worlds with agent players instead of scripted NPCs, so a new title feels alive from day one.We get into why agent-generated content could be the next step after studio-made and user-generated games. We also cover how agent populations might solve the chicken-and-egg problem every multiplayer launch faces, and what it means when your fellow players are relentless and never need sleep. Isaac walks through the NEXUS lab: a text-to-3D world builder for Roblox, an AI pipeline that turns ONE store novels into short web dramas, a loop-based tool for natural pixel animation, and Da Vinci, the internal multi-agent framework the team uses to ship.Then the conversation turns to infrastructure. We talk about why NEXUS went from 200 hand-defined agent roles to a much thinner harness, whether custom harnesses still matter now that Claude and Codex exist, and why the Kimi release pushed the company to start building its own small data center for inference. We close on context: how much of your life you should hand to an AI, what personalized meeting notes could look like, and why shopping agents may force every ecommerce site to rethink who it's designing for.Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmIssac Lee: https://www.linkedin.com/in/issacley/?locale=koTimestamps:[0:00] Cold open[0:57] AI in gaming at NEXUS[2:19] Text to 3D and the last 10 percent[4:38] AI companions and coaches[6:35] Agent players in MMORPGs[8:27] From UGC to agent-generated content[11:07] Agents that never sleep[12:58] Taming fuzzy prompts[19:24] Inside Da Vinci[23:10] Undercover QA agents[27:14] Do harnesses still matter[31:52] Self-hosting open models[41:33] Lab experiments and Bob[50:42] How much context is too much[54:22] Marketing and shopping agents

Caveman prompting has one rule: why use many words when few do the trick? It saves tokens on the way in and on the way out. Push it too far, though, and the output falls apart. So how far is too far? Nobody has benchmarked it yet, and that question opens our conversation with James Barney, Head of Forward Labs at MetLife.James spends his days connecting new AI capabilities to old business problems across dozens of regulatory regimes, and he still finds time to push code. He explains how the FinOps Foundation's AI working group took on the most basic question: which model for which workload, and why the answer always comes down to cost, speed, and accuracy. We get into Anthropic's launch pricing for Fable, why a million tokens is easy to price and hard to explain, and why every stakeholder eventually tells you what they really care about once you name the wrong North Star.From there it gets practical. Start with the smartest model, then step down and add harness until quality holds. Treat exploration tokens like local builds and production tokens like pipelines. Govern agents the way you govern people, with proactive blocks, reactive checks, and policy as code an agent can actually read.We close on a bigger shift. When a chat window can pull from every dashboard at once, do we still need dashboards? James thinks mostly not, with one catch he calls the latent shopper problem: some insights only come from browsing data you did not know to ask about.Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmJames Barney: https://www.linkedin.com/in/james-barneyTimestamps:[00:00] Cold open[01:03] Meet James from MetLife[01:11] What AI enablement means at a global insurer[03:27] Inside the FinOps Foundation AI working group[05:03] The caveman skill challenge[06:37] Why we need a CaveBench[07:43] Anthropic's Fable launch pricing[09:34] Planning for surprise model releases[11:45] Measuring AI value beyond cost[12:47] Finding your unit metric[14:15] Tying token spend to business outcomes[16:35] Explore first, then optimize[18:12] AI that tunes itself[20:13] Do R&D tokens count[23:24] When the experiment becomes the product[25:38] Personal agents and daily briefings[27:39] Governing agents that run just because they can[30:37] The layers of AI governance[31:56] Proactive and reactive guardrails[32:59] Policy as code for agents[34:41] Stop the click ops[35:31] Is the modern UI obsolete[36:36] MCP apps and chat as the new browser[38:07] How AI gathers data differently than humans[40:37] The latent shopper problem[42:12] Staying close to your data

Tool descriptions tell an agent what a tool does. They don't tell it how to use five tools together, in the right order, following your conventions. That gap is where this conversation lives.Filmed at AGNTCon + MCPCon in Tokyo with Ola Hungerford, Principal Engineer for AI Enablement at Nordstrom and a maintainer of the Model Context Protocol, who spent the last several months turning a pattern everyone was quietly reinventing into an actual MCP extension.Ola walks through what skills over MCP really means: the server stops being a pile of tools and becomes a distribution channel, handing the agent the instructions, workflows and knowledge it needs only at the moment it needs them. She explains why server instructions weren't enough, how progressive discovery keeps context from exploding, and why the same mechanism works for memory and preferences even when no tools are involved.Then it gets into the harder parts. What belongs in the MCP spec versus the agent skills spec. Why passing custom front matter through opens a rug pull and prompt injection surface nobody wanted. Where skills start to look like sub-agents, and why there's still no standard way to declare which servers a skill depends on. And the honest problem underneath all of it: how do you standardize something while everyone is still finding out what it's actually for, without breaking a hundred things the next time you change your mind?Timestamps:[0:00] Intro[0:21] AI enablement at Nordstrom[0:33] What skills over MCP actually is[1:29] The MCP server as a distribution channel[2:01] Server instructions versus skills[3:11] Distributing knowledge and memory[4:23] Progressive discovery explained[5:23] Where the idea came from[6:59] From draft to official extension[8:19] What early adopters changed[8:51] Front matter and custom metadata[9:57] Rug pulls and prompt injection risk[10:57] Will any of this get standardized[12:10] Marrying two very different specs[13:03] Skills as personas and sub-agents[13:53] The missing dependency standard[15:15] How the extension actually works[16:19] What harnesses still need to support[16:59] Consent and skill integrity[17:56] Where skills over MCP goes next[18:43] Why cramming 200 tools fails[19:41] Standardizing before you know the answer[22:18] Is git the wrong tool for agents[23:19] Picking tools for the actual persona[23:51] Trying to be less productive[25:48] The anxiety of idle agents[27:15] Why she keeps a robot on her desk[28:22] If the agent feels the friction, does it matter[29:41] Efficiency, waste, and caring enough[30:52] Letting an agent debug for you[32:03] Choosing your rabbit hole

A year ago, Palo Alto Networks built dashboards to track AI spend. Today those dashboards are useless, and the team that built them thinks that's the whole story.Recorded at FinOps X in San Diego, this conversation brings together Abhinav Lad, who leads cloud and AI finance at Palo Alto Networks, and Kuntal Patel, who runs the cloud engineering function behind it. They explain what happened when agents entered the picture, and AI stopped behaving like a service anyone could forecast.The short version: consumption went from linear to exponential almost overnight. Agents are goal-oriented rather than task-oriented, so they plan, call tools, verify, fail, retry, and keep looping until they hit the outcome, and every iteration is billable. So how do you run finance on top of that? Abhinav and Kuntal walk through the metrics that replaced their old forecasts: adoption rate, cost per user, AI as a percentage of revenue - and the budget limits that let engineering leaders choose between the newest model and a longer runway. They get into the open question of whether a cheaper model saves money or just burns more tokens thinking. They explain why an AI gateway became the control plane for cost and security at the same time, why retry caps belong in the design phase instead of the postmortem, and how FinOps starts to resemble product QA once the bill becomes the clearest signal that something is broken.They close on a warning worth sitting with: cost per million tokens is a number that means almost nothing on its own, and a value story built on it will point you somewhere you don't want to go.Palo Alto Networks: https://www.paloaltonetworks.comAbhinav Lad: https://www.linkedin.com/in/abhinav-ladKuntal Patel: https://www.linkedin.com/in/kuntalpatel35Alex Salkever: https://www.linkedin.com/in/alexsalkeverTimestamps:[0:00] Intro[1:00] Who runs FinOps for AI at Palo Alto Networks[2:10] Last year's AI dashboards are already useless[4:26] Agents turned linear forecasts exponential[7:27] Three traits that make agents expensive[8:34] The hidden bill: RAG, vectors and egress[9:16] Cost per user and adoption rate[11:21] Giving engineering leaders a budget[12:07] Using DORA metrics to prove value[13:53] Where DORA stops fitting AI[16:20] Does the cheaper model actually save money[17:57] Why you need an AI gateway[20:05] Inside Prisma AIRS[21:00] Three cost models for three use cases[22:52] Forecasting lessons from Electronic Arts[24:03] Runaway agents and endless loops[25:59] Capping retries before they burn cash[28:06] Writing cost policy at design time[29:01] When FinOps becomes product QA[32:17] Explaining AI spend to the C-suite[34:51] Valuing AI beyond engineering[37:04] Crawl, walk, run: where they are today[38:20] Why cost per million tokens is meaningless[39:26] Closing thoughts