Zillow’s real estate technology products touch roughly 80% of U.S. transactions each year, and the company has been using AI long before ChatGPT existed. At VB Transform 2026, Zillow SVP of Engineering Toby Roberts and Glean co-founder and CEO Arvind Jain described how they built an AI architecture meant to carry context across that entire journey — and why context, not raw data, turned out to be the harder problem to solve.
Why data foundations matter less than memory
Roberts said Zillow’s AI effort started where most enterprise AI efforts start, with the data itself. “We started with a large push around making sure our data did have the right foundation,” Roberts said. That meant a data mesh approach, clear data lineage and a governance structure with permissions and identity attached to the data itself.
None of that turned out to be the hard problem. The hard problem was building something that remembered where a customer was in their journey and carried that forward, no matter which surface they showed up on next. “This context layer has to live to be able to support you where you are at any given point in your journey,” Roberts said. Zillow chose to own that layer itself rather than depend on a single external chat interface, a decision Roberts said the team reached quickly once it looked at the shape of a real transaction rather than a single conversation.
Related: Intuit Rebuilds AI Agent Orchestration System
Centralizing context to cut costs
Zillow built its own harness rather than route customers through a single model API. The team drew on 20 years of machine learning history behind products like Zestimate, leaning into smaller, task-specific fine-tuned models instead of one general-purpose model. Internally, that harness runs alongside Glean. Roberts said Zillow now has thousands of Glean agents in production, handling repetitive tasks with tens of thousands of executions across the company.
Glean’s pitch is centralizing that integration work once, through the Glean MCP gateway, rather than letting finance, legal and marketing each rebuild their own connections to the same systems. That centralization is also a cost lever. Jain pointed to two mechanisms: model routing, which sends most tasks to smaller, cheaper models instead of defaulting to frontier models, and precomputed context, which avoids an agent burning tokens assembling its own context from scratch. “Claude is also very slow because the first part of assembling that context actually takes forever,” Jain said. Routing that request through Glean instead, he said, can cut token consumption by as much as half.
Roberts said Zillow’s ability to credibly attribute a 40% increase in shipped code to AI adoption rests on a DORA metrics baseline the team put in place years earlier, not on the AI rollout itself. This reliance on historical baselines offers a stark contrast to the “move fast and break things” mentality that often plagues enterprise AI pilots, where teams chase immediate wins without the infrastructure to measure long-term impact. Without that baseline, a company cannot distinguish between genuine productivity gains from AI and the natural variance in software development velocity. It suggests that successful enterprise AI integration is as much about organizational discipline as it is about model performance.
Related: Enterprise AI leaders pull ahead of peers
Practical takeaways for enterprise AI
Across data, cost and permissions, the session offered a few practical takeaways for enterprises building agentic AI on their own systems. Roberts said Zillow chose to own that layer itself rather than depend on a single external chat interface. Jain’s core argument for Glean’s platform is that duplicated integration work across finance, legal and marketing teams is a hidden cost most enterprises haven’t accounted for.
Even with a permissions-aware context platform in place, Zillow layered hard rules and a standing compliance check on top for its most sensitive categories, rather than trusting the architecture to handle it automatically. Model routing and precomputed context were the two mechanisms Jain pointed to for cutting AI spend, both aimed at reducing wasted token consumption rather than adding new capability. “Models by themselves are not enough to bring automation with AI inside your enterprise,” Jain said. “You do have to connect it with your enterprise context.”
Leave a Reply