For financial services companies, good incident management is absolutely critical—maybe more so than in other industries. So, for Michael Cullum and his team at Bud Financial, the choice to build an incident response felt right for them in the moment.
As the VP of Data and Engineering, Michael oversaw the creation of a Slackbot to streamline incident response and get some guardrails in place. “It'd been one of those things that we created in a few days and tacked on bits later on. It did quite a lot of stuff,” says Michael.
For them, it solved the immediate problem of not having a sensible, consolidated, and predictable way to declare and manage incidents.
But very quickly, Michael and the team came face-to-face with the myriad limitations that come with building your own response tooling. Soon after, they would adopt incident.io to replace that internal Slackbot and save money, resources, and headaches in the end.
The end result? A total transformation of the way they managed incidents.
Living with the constraints of a Slack bot
The first attempt at creating a dedicated incident management process came with the building of said Slackbot. For Bud, it was quick, cost-effective, and easy to build.
Its primary function was declaring incidents and creating an incident channel to manage them. But Michael started to notice that many roadblocks came with it. “There are a lot of constraints around Slackbots. It's very easy to do something quick and dirty, but then it doesn't always work,” says Michael.
And when responding to an incident, the last thing you want is for something to go wrong.
And even if you end up creating a bot you’re happy with, once you try to expand its scope beyond incident declaration, things start to fall apart even more.
“It's easy and simple to do the basics, but then it starts to go wrong very quickly.”
“Doing what we’re good at and what’s core to our mission”
When you build your own tool, the work doesn’t end once the final pull request is merged. You have to deal with all the bugs, maintenance, and upgrades after the fact, too.
This can amount to a lot of extra work for teams already stretched thin working on loads of other projects. In the end, many companies who go the build route come to the realization that the work that they’re doing is better left to someone who specializes in it.
This is especially true when it comes to a critical function like incident management.
“I'm a big believer in, as a company, you should do what you're good at. You should do what’s core to your mission and focus on how you're adding and creating value,” says Michael.
“At Bud, that's getting financial insights from data. Not maintaining an incident management Slackbot that integrates with PagerDuty, GitHub, links to post-mortems, and nudges people to take certain actions. Every minute we spend fixing a bug is time we're taking away from adding value to our customers.”
Better processes for better customer outcomes
For Michael and his team at Bud, the decision to build their own tooling, at first, made the most sense to them. But soon after coming to terms with the limitations of having a Slackbot for incident response, they realized that the issues extended way beyond that—namely to processes.
When someone on his team came across incident.io, Michael realized it was just the solution they needed to put a robust process in place to manage incidents—from declaration to post-mortem.
“When an incident happens, you're not quite where you want to be. It can be a stressful situation for you, your clients, and your customers. You want to spend your time focusing on, ‘How do we improve our process? How do we improve our value? How do we offer the best service during these instances?’” says Michael.
That's where you want your focus to be. Not on having to figure out how to build the tool that will do the thing you want. It's about being able to give you that flexibility to make those processes better so that you can deliver value and quality to your customers and clients.
How incident.io made the decision to buy the best choice for Bud in the end
For many companies like Bud, the choice to build their own software felt like the most sensible one at the beginning.
But once the shortcomings started to compound, like limits on functionality and cost of maintenance, it became clearer that buying software would not only remove lots of hurdles, it would also lead to cost savings down the line.
For Bud, everything changed once they decided to onboard with incident.io. Since then, they haven’t looked back.
“What I saw was the opportunity for a tool that could go well beyond something that solves the problem of being able to declare an incident and creating a Slack channel,” says Michael.
“But something that could actually help us improve our incident management and response processes with further capabilities that come down the line. What we've seen over the past few years is exactly that.”
Flexibility to manage incidents better and smarter
One of the many unsung heroes of buying software is the flexibility that comes with it. And when building out your go-to incident management processes, this flexibility makes all the difference as you scale.
“One thing that’s been really nice is the creation of powerful and flexible functionalities like Workflows and Catalog,” says Michael.
The benefits of having access to a codebase they built with their Slackbot are far outweighed by the flexibility of having features like these.
"The usefulness of these features has allowed us to almost ignore the fact that we don't have access to the code of our Slackbot anymore.”
Need support? We’ve got you covered
Even if you build an internal tool that you’re happy with, one of the aspects you’ll be missing out on is high-quality support when you need it. While you may have team members who are willing and able to pitch in when needed, their focus should be on delivering value, not fixing internal tooling.
For Michael and the team at Bud, being able to pass along feedback and ask for help whenever they need it with incident.io has not only been meaningful, it's proven indispensable.
“With incident.io, we're dropping feedback into a shared Slack channel with the team. Every time we’re giving that feedback, it’s acted on. With incident.io, you’re almost obsessed with making the product better,” says Michael.
If I come with a bit of feedback and say, ‘Hey, the team is struggling with this,’ it gets fixed.
Workflows that make incident response easier
When they built their Slackbot, Michael and his team only had access to a tool that declared and managed incidents, with a lot of limitations. But now, with incident.io, they have a host of features that they can look towards that streamline and dramatically improve their incident response processes.
“Right now, if a member of our client services team says, ‘If there's an incident involving one of my clients, I want it to send me a text message to alert me,’ they can just go in and add that. Previously, that just wouldn't have been possible or would have required development time for that one individual task,” says Michael.
The Workflows feature gave us the ability to iterate our processes and bring the right information to people at the right time. Building something like this was just never going to happen. Whereas with incident.io, you've got tens of product engineers focused on creating a better product for incident response every single day. Because of that, you're always going to do a better job than we would.
Buying? The ultimate return on investment
In the end, the decision to buy incident.io to replace their internal Slackbot was not only the right decision for Bud, but it ended up being transformational.
“For me, it's about three things: One, I like to call focus. Because, ultimately, yes, there's a monetary point there. But being able to focus our platform and infrastructure development teams around problems that are specific to us, it's just one less thing that we have to worry about,” says Michael.
“Second, in terms of financial impact, we've definitely spent less on incident.io than we have on just maintaining the tool. And that's a dollar-for-dollar easy comparison to make. Third, incident.io has enabled us to have a better incident response process. What does that mean? Better service to our clients, which means better NPS. It means better net revenue retention and client satisfaction, which ultimately feeds right down into revenue and our P&L statement. Can't really argue with that.”
