Managing what you can’t predict: the development lifecycle for agentic applications

Adding an LLM to your application takes an afternoon. What comes after is where it gets interesting. The moment you introduce a model, you introduce uncertainty into a system that used to be deterministic, and most of our engineering practices quietly assume determinism. Everything we know about good software development still applies. Version control, testing, CI/CD, and code review don't go away. But they're no longer enough. Harness engineering, the scaffolding you build around the model, is part of the answer, but it only covers construction. It doesn't tell you how to find out what works before you build, how to prove it keeps working, or how to know it's behaving once it's live. This talk covers the full lifecycle, from offline experimentation, through evaluation as your real quality gate, to production monitoring. Each of those asks something of you that traditional development didn't. The talk draws on hands-on work building agentic applications, including the practices that proved their worth and the questions that are still open.