turbopuffer builds a search engine used by Cursor, Linear, Anthropic, and Ramp, some of the fastest-growing AI companies in the market. Their object-storage-native architecture means uptime is the norm, four nines of database uptime so far this year, but on the rare occasion they go down or degrade, some of the world's most widely used AI applications go with them.
As CTO Nikhil Benesch puts it, "If you don't run your own software in production, how do you know whether you're designing it correctly? When you ask the engineers writing the code to also carry the pager, they learn: when I write code like this, that's where the incidents pile up."
The trade-off is straightforward: the people building the product are also the people responding when it breaks. Nikhil thinks about it in budgets.
Every incident in the middle of the night wears the team down a little. There's also a trust budget with our customers. We pay close attention to both.
A typical incident at turbopuffer started with the primary on-call triaging the alert alone, deciding whether to pull in a secondary, then starting the investigation cold: searching logs, checking recent deploys, and pinging whoever might have context while the incident was still running. With no dedicated SRE to lean on, how fast the process went came down to whoever was holding the pager.
turbopuffer was already using incident.io’s on-call and response products, and the quality of those products gave Nikhil confidence that their new Investigations product would be just as strong. He'd been watching the AI SRE space for something to help his small team work through incidents faster, so when incident.io launched Investigations, he took notice. Unlike the standalone tools he'd seen, this was an agent built into the response workflow he and his team already used.
Nikhil reached out through the support channel to learn more. A single 30-minute call with the incident.io team convinced him and Ben Linsay, one of the infra engineers, to sign turbopuffer on as a design partner for Investigations. Nikhil framed it as a bet worth making: "We don't have the luxury of running three POC processes, so we needed to bet on the right course the first time. We came out of that conversation thinking, we want to bet on these guys."
When we use it, we can really tell this is something built by people who operate software every day. The incident.io team viscerally gets the value of responding to an incident quickly.
As a design partner, turbopuffer’s feedback goes straight to incident.io’s engineers, who are quick to turn it into shipped improvements.
Building something in-house was a real option for a team like theirs, so the question became what incident.io offered over a DIY setup.
"The question we ask ourselves is: what's the edge this product has over just pointing Claude Code at a Datadog MCP server and a Slack channel? That's where incident.io is building something differentiated: the rules and scaffolding on the back end, and the fact that you actually keep track of what worked in the past. That's the edge, because you get to do this across every company running software in production, not just us."
Nikhil Benesch, CTO, turbopufferOnce they committed, turbopuffer moved fast. Nikhil set Investigations up himself in a couple of hours, turned it on across every incident at once, and it's been running ever since.
When turbopuffer's billing service broke one morning, Investigations pinned down the cause within minutes, before any of the responders knew what had changed. The service runs in the background, aggregating usage events from database servers and converting them into billable charges stored in MySQL. If that aggregation stops running, the company can't bill its customers, so the team getst paged.
The alert indicated the billing service was running out of MySQL connections, with no obvious reason why. The cause Investigations surfaced: a pull request merged earlier, intended to improve performance, had bumped the connection pool from around 10 to 50. When multiplied across dozens of servers, the change required hundreds of new MySQL connections—more than MySQL had available. Investigations also pointed the team to the engineer who had the most context on that change, so they could loop them in and decide whether to revert it or ship a fix.
None of the engineers initially pulled into that incident knew the change had gone out. We would have eventually tracked it down in GitHub. But Investigations can search through every merged PR much faster than a human can, and it focused its search on the PRs that landed right before the errors started. That's the kind of correlation that takes a human 30 minutes to spot—or if you're unlucky, hours.
turbopuffer is hiring fast, and newer engineers need time to get comfortable operating the infrastructure. Investigations shortens that ramp by pointing them to how similar incidents were resolved before. When it tells them "this is the same thing that happened two nights ago, here's what was done to resolve it," what Nikhil calls "waves of relief" follow. As he puts it, Investigations doesn't need to root cause an issue to be useful; it just needs to save time. If it can tell a half-asleep responder that a problem is a recurrence from two weeks ago and the fix was to wait ten minutes, that alone is a gift at three in the morning.
Every minute you save is so valuable. Even if it's only 10 or 20 minutes, those are incredibly high-stress minutes. They're worth a lot more than your average 10 minutes.
Ben Linsay sees the same benefit from the on-call seat: Investigations gives the team a quicker understanding of what's happening, accelerating how fast an on-call engineer can find a problem, internalize the root cause, and turn it into the work needed to fix the system. “Even when it's just saving you half an hour, it reduces the number of interrupts the on-call has to deal with. It's giving the on-call superpowers.”
Ben has also started running a monthly incident review through incident.io's MCP server, using Claude Code to compare incidents across months and pull out themes the responders might have missed in the moment. For a lean team without a dedicated SRE function, it provides a lightweight way to review patterns across incidents.
Investigations lets us get more out of the on-call team we have. It's stress relief. It allows us to support more customers with a smaller team.
turbopuffer is a small team, and the search engine they run sits underneath some of the most demanding AI products around, which makes on-call heavy work. Every minute Investigations takes off firefighting and root cause analysis is a minute less spent in the stressful part of an incident, and a minute the team gets back to building something faster and more reliable. As a design partner, they're helping shape where Investigations goes next: sharper accuracy on the incidents it can already crack, and more visibility into how it reasons.
