Stay up to date

We believe everyone should be paid for being on-call, and that the fairest way is to pay for the inconvenience. Practically speaking, that means paying an hourly rate for the time someone spends on call.

That's what we do at incident.io and we know that the calculations can get pretty complex. So we figured: what if a product could 'just do it', and you could shelve that dodgy spreadsheet or CLI script once and for all.

Introducing: the on-call calculator

You can use the calculator to generate reports based on your on-call pay configuration, using the schedule data from inside PagerDuty. The calculator supports weekly rules (e.g. extra pay outside working hours, or on the weekends) as well as holiday rules (think double-pay for Christmas).

Even better, your responders can also see a breakdown of what they’re being paid for each shift, making it super transparent for everyone.

If you're interested, you can try it out here.

n.b. If you're interested in on-call pay in general, we've just published the results of our on-call survey.

New workflow triggers

We've create three new workflow triggers, to help you build more flexible automations:

1. When an incident's severity changes
2. When an incident's status changes
3. When a custom field value for an incident changes

This lets workflows handle cases like "every time the severity becomes critical", or "every time the list of impacted products changes".

For example, this workflow would text me every time the incident severity increases to a value of Major or Critical. It would not text me if the severity decreased (e.g. from Critical to Major).

What else we shipped

• 🆕 You can now clone a workflow so you don't have to start from scratch every time
• 💅 You can now use variables in the subject-line of emails (not just the body)
• 🐛 You can now select labels when exporting follow-ups to GitHub
• 🐛 We removed some erroneous full stops in our PagerDuty messages
• 🐛 We now support organisations whose Zoom integrations require numeric-only passwords
• 🐛 We fixed a bug where users without Slack display names wouldn't show up on announcement posts
• 👷 We've done some work to make our API faster (and the web UI along with it)

We know you rave about our workflows - their flexibility lets you configure processes just to your liking. But we'll admit, it was a bit annoying when you wanted to exclude old incidents, because you had to remember to add the filter for "where status is not closed". If you didn't, you might see some weirdness like getting prompted to "add a post-mortem" when changing the custom field on an old incident 🙁

So we've given you more power! When creating or updating a workflow, we'll prompt you with this modal:

It lets you declare which incidents your workflow should run on. Not only does this make it easier to exclude closed incidents, but it allows you to create workflows that will only run on future incidents!

Additionally, we wanted to help with edge cases that arise when you update a workflow whilst runs are still in-flight. This might happen when theres a delay between steps. We'll let you state whether the in-flight runs should be left to run as planned, or should continue running based on the new version of your workflow, or be cancelled entirely.

One more thing

We're very excited to have just dropped our Practical Guide to Incident Management! Every company needs a plan for when things go wrong. So here it is – our years of collective knowledge & experience distilled into a practical guide. Enjoy!

What else we shipped

• 💅 We now support variable interpolation in the post-mortem markdown editor
• 💅 We'll now post in an incident channel when an attached github PR is reviewed
• 💅 We added an option to "reassign action to me" for assigned actions
• 💅 When you an update an incident, we'll now show how long it is until the next update is due
• 💅 We added the incident reference (eg. INC-123) to the incident slack channel topic
• 💅 We added the Jira ticket URL to the exported CSV of incidents
• 🐛 We started formatting actions consistently
• 🐛 We started showing real names for observers in a post-mortem when display names are unavailable (thanks Bitvavo!)
• 🐛 We fixed a bug with updating "Someone added an action" messages
• 🐛 We stopped posting duplicate workflow actions to a channel

It's conference season, and the whole team has been out: speaking to customers, handing out swag, giving talks and of course ... grabbing lots of swag ourselves.

First, we headed down to LeadDev London, just a ten minute walk from our new office in Old Street. If you were lucky enough to be there, you will have seen us handing out some classic incident.io swag, including:

• Our misprinted limited edition t-shirts!
• Fire-sale incident.io socks
• And some beautiful notebooks

At LeadDev, the one-and-only Lisa Karlin Curtis presented "Using Incidents to level up your teams" – this talk will be available online on 17th June, so watch this space!

Not content to stop there, we were also at PlatformCon 2022 where:

• Lawrence presented "Growing into platform engineering"
• Lisa (overachiever, this one!) presented "The power of transparency: building trust when things go wrong"

We had a blast catching up with customers, and if you missed out on any swag, be sure to grab us quickly at the next one (or Tweet us, and we may just send you what you missed...).

What else we shipped

• 📚 The Intercom help docs are now available from within the dashboard
• 💅 You can now filter the incidents list by multiple Incident Types
• 💅 The summary has been added to the Create Incident modals
• 🐛 Handled race conditions with multiple people accepting/declining/merging triage incidents at the same time
• 🐛 Fixed a bug when escalating to a PagerDuty service with 100s of available responders
• 🐛 Incident channels won't be archived if there's still activity, even if the incident has been closed

It's been a couple of weeks since our last changelog - we had a team offsite in Portugal for 4 days and managed to avoid /incident sunburn (mostly). We're back with a bumper update!

Workflows are a powerful way to help people follow your incident response process. One of the most-requested improvements was to allow you to delay the workflow run after the trigger happens. For example, to remind people to write a post mortem a few days after an incident is closed.

You can choose whether the workflow should run even if the conditions stop matching, or if we should cancel it. This lets you avoid annoying reminders like "please write a post-mortem" if one is already attached!

What else we shipped

• 🆕 We launched Incident Types last week
• 🆕 Workflows can now set a condition on "outstanding action count", so you can set reminders to complete actions
• 🆕 We launched our customer referral programme
• 💅 When deleting an Incident Role or Custom Field, we'll warn you if it's used in any Workflows
• 💅 You can link directly to one of the tabs on the Incident Homepage
• 💅 The timeline on the Incident Homepage can be filtered by item type
• 💅 The "can this be closed?" nudge in Slack now has buttons to dismiss the nudge and to provide an update, as well as to close the incident
• 💅 Post-mortems can be exported as Markdown, as well as HTML - drop us a message if you'd prefer Markdown!
• 🐛 Fixed a bug which caused post-mortems not to get copied to the clipboard in Safari
• 🐛 Custom field descriptions are now visible for single-select fields

Responding to incidents can be stressful. You've got 14 tabs open with some logs, a Sentry, a trace, some database queries, and a couple of GitHub PRs. It's hard to keep track.

Attachments is our first step to solving that problem.

Attachments allows you to connect something from another system into incident.io. We're not just talking about putting a link on the timeline though, we think you deserve better than that.

Currently, you can attach PagerDuty Incidents and GitHub Pull Requests to an incident. But don't worry, there's more to come.

To attach something, you simply need to paste a link into an incident channel:

You can view attachments in Slack (via /incident attachments) and in the Web UI:

Once it's attached, we'll help you out wherever we can.

We'll comment on the PR (if it's in a private repo) with a connection back to the incident, so it's easy to find in the future

When a Pull Request gets closed or merged, we'll let you know in the channel

To start using the feature, paste a GitHub link into an incident channel and we'll do the rest!

What else we shipped

• 💅 The follow-ups page now only refers to follow-ups, not actions (thanks Lob!)
• 💅 The 'generate post-mortem' button now links direcly to a modal to make the flow nice and smooth
• 💅 When we nudge you to update the summary, if you do, we'll remove the message so it doesn't clutter the channel
• 💅 Long action descriptions will render better in the web UI
• 🐛 If you uninstall Zoom, you can now install and enable auto-call with Google Meet
• 🐛 We won't ask if you want to close the incident while you're actively talking in the channel
• 🐛 Links will format more nicely in PagerDuty auto create messages

For many of you, incident.io sits on top of PagerDuty. You get paged. You check if the error is concerning. If so, you pop Slack open and declare an incident. Then you paste a link to the error in the incident’s channel. Rinse and repeat for every alert. If you suffer through some flavour of this cumbersome, repetitive flow, our latest release is for you!

To save you doing all that manual work over and over again, we can now create incident.io incidents straight from your PagerDuty alerts.

So how does it work? You start by choosing the type of PagerDuty incidents you care about, i.e. the services and urgencies for which we should auto-create incidents. Whenever we see a PagerDuty alert that matches that configuration, we’ll automatically create an incident.

Importantly, that incident will be created in a newly-minted triage status. This is to give you the time and space you need to investigate the alert. When you have a clearer idea of what’s going on, you’re ready to move the incident out of triage. We’ll have posted a message in the incident’s channel, with three possible outcomes and next steps:

1. “A new incident” means that we’ll update the status from triage to investigating - from here on, you can keep managing your incident in this channel like any other incident.io incident.
2. “Part of an existing incident” will let you point to another incident that this alert should be merged into. We’ll take care of archiving the channel for you, and making sure that incident isn’t reflected in your Insights.
3. “Not an incident” will tell us to dismiss this incident. Again, we’ll take care of archiving the channel for you, and making sure that incident isn’t reflected in your Insights.

All the PagerDuty alerts linked to your incident will be ready for you in the “Attachments” tab on the Incident Homepage. To save you from more repetitive manual clean-up, when the incident is closed, we’ll also go and resolve these attached PagerDuty incidents for you!

And for the final flourish of magic: while you’re working in an auto-created incident channel, we’ll notify you about similar PagerDuty alerts before creating new incident channels. No noise.

Happy days!

What else we shipped

• 💅 Added the latest update to the "Recap" slack modal
• 💅 Split the concept of "creator" (eg. API key) from "reporter" (eg. Milly) in the API
• 💅 Show more information about recently created incidents when creating a new incident
• 🐛 Stopped listing "Incident Reporter" twice when assigning roles in workflows

Today we're releasing our API to the world. It's been in early-access for a few weeks and it's been amazing to see what people have built with it, and we can't wait to see what other integrations you'll create.

You can use our API to declare incidents from other tools, export all your data, and configure custom fields, severities, and roles automatically.

See the full details on our blog, or dive into the API reference. We'd love to hear what you're building in the #api channel in our Slack community!

What else we shipped

• 🆕 Web: You can generate a post-mortem document for an open incident
• 💅 Slack: The messages we post when we've escalated an incident make it clearer to see who's been contacted
• 💅 We show your real as well as display name when selecting a user, to make it easier to find the right person
• 🐛 Fixed a race condition when exporting incidents as a CSV

When it comes to private incidents, our top priority is keeping private incident data safe. Being able to harness the power of our whole product during a private incident is a close second.

It's now possible to build workflows that act on private incidents. We'll only let account owners make changes to these workflows, to make sure that other users can't use workflows as a way to gain access to sensitive information.

Along with that, we've also added a condition so you can choose to target only private incidents with your workflow (if you'd like).

You can learn more in our help centre.

What else we shipped

• 🆕 Web: You can now assign actions and follow-ups to a user (or role) when configuring workflows.
• 🆕 Web: You can specify the urgency when using a workflow to escalate to PagerDuty or OpsGenie.
• 💅 Web: You can search for users using their full names (not just their display names) in the web UI.
• 💅 Web: You can now choose shorter auto-archive windows.
• 💅 Web: Incidents with long names look better in the UI.
• 💅 Slack: It's now clearer that we won't announce test incidents anywhere.
• 💅 Slack: We've stopped tagging people in all our updates in Slack, so you can still see who did what, but without the unecessary noise.
• 👷 Web: Workflow dropdowns with lots of options are now much snappier.

We know you love exporting follow-ups into your favourite issue tracker. So much so, that a number of you recognised that we don’t don’t sync changes quite fast enough!

We’ve doubled down on our Jira Cloud integration, meaning we now listen for issue updates and sync these to follow-ups immediately. So next time you update the Jira issue update status to “Done”, this will also get ticked off in incident.io 💪

What else we shipped

• 🆕 We now aggregate related items on the timeline
• 💅 Update reminders are more reliably cancelled if you post an update in time, and are ephemeral so do not spam the channel
• 💅 Instead of being truncated, long incident updates are continued in a thread
• 💅 We renamed the Insights “Scoreboard” to “Responders”, because firefighting should not feel like a competition
• 💅 When a timeline item (eg. Milly was assigned to X) was generated by a workflow, we now point to the workflow that produced it
• 💅 We now stop unfurling message links by default, for cleaner, happier channels
• 🐛 We started showing links to issue trackers (eg. Jira) properly in the postmortem (thanks Snyk)
• 🐛 We removed the ability to filter workflows based on the incident URL, because that’s just confusing
• 🐛 We stopped showing the “I can take this” button on follow ups, because the incident has already ended
• 🐛 We stopped long custom field links spilling out of the field (thanks AccuRx)
• 🐛 Now when you create a new announcement rule, it’ll post a message in the channel saying so
• 🐛 We fixed an issue with not being able to create links in the compose message flow (thanks Ascend)
• 🐛 We now allow emojis to appear in actions that are created from Slack messages (thanks Aiven)

Time zones are confusing enough as it is - I should know, I was very confused when the clocks went forward on Sunday - so we weren't helping things by displaying timestamps on the dashboard in UTC for those of you in other parts of the world. Now if you hover over a timestamp (or tab to the timestamp with the keyboard), it'll display the timestamp in both UTC and your local time, in a format a bit more familiar to you (like MM/DD/YY for you folks in the US).

What else we shipped

• 🆕 Web: Timeline items now link back to their Slack posts
• 🆕 Web: We'll now render a timeline item when you rename an incident
• 🆕 Web: We now create timeline items for all images shared in an incident channel, not just pinned ones
• 🆕 Slack: You can now assign the lead role for your incidents using the role name you defined (not just "Lead")
• 🆕 Slack: There's now a button to post all the open actions to the incident channel in the incident actions menu (/inc actions)
• 💅 The incident summary now supports Slack mrkdwn across web and Slack.
• 💅 Slack: We've put back the prompt to add a summary when there isn't one, but it's less annoying now
• 💅 Web: We now show the incident status in the list view
• 💅 Web: You can now tag @here and @channel in your Workflows in the Slack send message steps (thanks Yoco)
• 💅 We've refined some of our Slack posts and modals to make them a bit more consistent
• 🐛 Web/Slack: Fixed an issue where some Jira dropdowns (e.g. Priority) weren't populating when exporting an issue (thanks Aiven)
• 🐛 Web/Slack: Un-pinning and re-pinning messages will now recreate timeline items (thanks again Aiven!)
• 🐛 Web: Fixed the "Request access" dialog not showing up when you tried to access an incident you didn't have access to
• 🐛 Web: We're now displaying a more helpful error if you try to connect Github when you aren't an organization owner
• 🐛 Web: Fixed a bug where exporting a follow-up to Linear would lose the description (thanks Render)

Look, sometimes you can't get to solving all your follow-up actions right away, we understand. So maybe your follow-ups page was looking a little full? It's ok, we've got you, we've added the ability to filter the follow-ups page the same way you can filter the incidents page.

One more thing

We also added a workflow step to invite a user from a PagerDuty escalation policy, and assign them an incident role!

What else we shipped

Actually 17 more things...

• 🆕 Include pinned images in post-mortem PDFs
• 🆕 Include completed action titles in the timeline
• 🆕 Show recent incidents in the create incident modal, to help avoid creating duplicate incidents
• 🆕 Show shorthands in the /inc help info, so you can learn how to move faster
• 🆕 Show the reasons we need to reinstall on the reinstall page
• 🆕 Show absolute timestamps in the timeline
• 💅 Update setup/login step to hide the sidebar
• 💅 Make incident update -> closed not do anything until the incident is closed
• 💅 Auto-focus "title" when opening the create action modal
• 💅 Make "description" optional when exporting an action
• 💅 Hide Assignee when exporting to Jira
• 💅 Allow toggling private incidents on follow-ups page
• 💅 Relax limit on decision title flow length
• 💅 Allow reversing timeline order on incident home page
• 💅 Handle "action already exported" gracefully from Slack
• 💅 Show "not set" for empty workflow values when viewing workflow configs in the web app
• 🐛 Fixed an issue where the login screen had multiple headers on mobile

Custom fields are a key part of incident.io, helping you catalog past incidents, analyse trends and integrate with workflows and announcement rules.

Today we're releasing support for three new types of custom field:

Text

Supporting freeform text values, create fields that record the impacted customer, or perhaps any third-party services involved in the incident.

Number

Especially for regulated industries, you often want to track the number of customers affected by an incident, or the amount of money involved.

Numeric fields support integer and fractional numbers, which you could use to create:

• "Customers affected", tracking the number of customers affected by an incident, which you can use to filter the incidents page and CSV export
• "Monetary impact", for the financial value associated with an incident, which you can use to build a workflow that emails senior leadership whenever it exceeds a reporting threshold

Link

Imagine your company encourages creating a separate public post-mortem/root cause analysis, and you'd like to track that doc alongside the incident.

Or perhaps you create GDrive/Dropbox spaces for each incident, containing files used during the response.

You can now create a link custom field which is synced with a Slack channel bookmark on the incident channel, helping responders find the links during the incident and whenever anyone visits an old channel, too.

What we shipped

Custom fields is just one big change that went out this week, while another is our totally redesigned website!

Thanks to @jamesm for his amazing design work.

And beyond that, here's the full list:

• 🆕 Edit incident summary and name from the incident homepage
• 🆕 Support for an "is all of" filter for multi-select custom fields in the incidents dashboard page
• 💅 Exporting follow-ups to Linear now assigns the correct "created by" name
• 💅 Users are sorted alphabetically in the Slack role assignment modal
• 💅 Post-incident feedback is now shared in a thread instead of the channel, in an effort to reduce noise
• 🐛 Fixed an issue with Jira auto-export where the epic name might be incorrect

We think incident response should be open for everyone, but there are some things that are best kept within a smaller group.

We've introduced roles within incident.io, so some organisation-level settings can only be changed by selected Administrators.

Users can have one of four roles: Viewers, Responders, Admins and Owners.

Viewers can:

• View incidents in Slack and the Web Dashboard
• Declare incidents

Viewers will automatically become responders when they get involved in an incident.

Responders can:

• Use incident.io via Slack and the Web Dashboard
• Create and edit Workflows and Announcement Rules
• View organisation-level settings

Admins can:

• Do everything a responder can, plus:
• Edit organisation-level incident.io settings
• View and edit billing settings

Owners can:

• Do everything an admin can, plus:
• View all private incidents

If you'd like to take a look at your Users, check out our shiny new Settings page.

n.b. Until March 1st, this feature is opt-in for existing customers.

What we shipped

Here's what else we've been up to this week:

• 🆕 We've built a new pricing model, moving to value-based pricing
• 🆕 You can now invite users to incident.io via the dashboard, from the Users Settings page
• 💅 You can now tell Statuspage to post your updates to Twitter via incident.io
• 💅 We now pre-fill the Statuspage update title, so you don't have to write it again
• 🐛 If you try to install us into a personal Github account (which we don't support), we'll let you know that something's gone wrong
• 🔜 A whole lot of work on custom fields - watch this space...

❤️ Happy Valentine's Day from all of us at incident.io ❤️

It's great to be able to track incident follow-ups alongside other work in your issue tracker of choice.

As a festive treat, we're announcing a new partnership for us at incident HQ. We've supported Jira Cloud for a while, but today we introduce support for Jira Server!

As with all our issue tracker integrations, once connected, you can export follow-ups to Jira Server from the web dashboard. We'll then keep the incident.io follow-up 'in sync' with your Jira board, so updates in Jira will be reflected inside incident.io on the follow-ups page.

You'll also be able to view more details on the insights page, to understand how your team is managing follow-ups.

What we shipped

Here's what else we've been up to this week:

• 🆕 You can now configure a minimum severity on the default announcement rule
• 🆕 We can now send SMS messages to US phone numbers
• 💅 The Incident Home modal in Slack now shows reminders of the slash commands to help
• 💅 It's now easier to claim the incident lead role when a lead isn't set
• 💅 We display a clear banner to indicate when you're logged in to our Slack Community instance of incident.io
• 💅 It's now possible to ask Statuspage to share your updates on Twitter when updating it from Slack.

Finding patterns in why things go wrong can be hugely powerful in working out where to focus your limited resources. It can also be a huge pain to dig into this data.

Today we're releasing a revamped Insights section that gives you far more access to dive into your incident response data.

We're starting off with three sections:

1. Incidents

This is where you'll find high-level metrics like median time-to-fix, and a chart of incidents reported by severity:

2. Follow-ups

As the old saying goes, "fool me once, shame on you, fool me twice, shame on me." Taking the steps to learn from every incident and prevent the same issue coming back again is crucial. Here, you can see the completion rate of follow-up actions over time, and break them down by severity. You can also see how many incidents are having a post-mortem written.

3. Scoreboard

Everyone loves a little healthy competition. This tab shows you top incident responders: people who report the most incidents, lead the response, or just come to help out.

What we shipped

Here's what else we've been up to this week:

• 🆕 We've introduced "owner" and "administrator" roles: more on this next week
• 🆕 When you make an incident channel private, we'll make the incident private too
• 💅 The Incident Home modal in Slack now has emojis to help you find the right action faster
• 💅 The person that posted the incident update is now available as a variable in Workflows
• 💅 We only show 5 open incidents on the dashboard home page
• 🐛 Emails sent from Workflows now have better spacing around variables and render more cleanly in Outlook

We strongly believe that incidents should be public by default, and that transparency is an important foundation of any good incident process.

However, we recognise that there's clearly exceptions where public incidents aren't appropriate, such as:

• Security incidents - e.g. data breaches or vulnerabilities
• Compliance incidents - e.g. to comply with anti money-laundering regulations
• HR incidents - e.g. to handle a sensitive situation in your team

We've had a lot of requests from our customers to better support this use case, so we've built Private Incidents!

How do Private Incidents work?

When creating an incident after enabling Private Incidents, a user can choose to make it private.

If an incident is private, we will:

• Create a private Slack channel to manage the incident (as opposed to a public channel, like we do today)
• Invite the reporter to join the new Slack channel

We will not:

• Announce the incident in the #incidents channel, or any other channels
• Make the channel visible to everyone: unlike public incidents, only people with access to the Private Incident will be able to see it in the web dashboard (including any follow-ups)
• Run any workflows, or automation features like ‘incident watchers’

Controlling access

Any member of a Private Incident can invite someone else, either by inviting them in Slack or via the web dashboard.

You can also revoke someone's access if you need to: everybody makes mistakes!

How can I use it?

Head over to your Settings and enable Private Incidents.

Now, whenever you declare an incident, you'll be asked whether to keep it public, or make it private.

---

As we've had so many requests for this feature, we're very excited to be releasing it! Take it for a spin, and let us know your feedback by joining our Community and sharing in the #feedback channel.

What we shipped

Here's what else we've been up to:

• 🐛 Fixed a bug where the 'I'm on it' buttons wouldn't work if you created multiple actions from a workflow.
• 🐛 You can no longer update your public statuspage from a test incident by going via Incident Home.
• 🐛 It used to be possible to get into a broken state when uninstalling the GitHub integration: this shouldn't happen any more.
• 🐛 The incident name is no longer mandatory when declaring an incident from the dashboard.

It's the new year, and we're all ready for a change. Some of us have been tidying up and clearing out the cupboards - our CEO Stephen got rid of 6 bags of clutter over the weekend.

The same goes for the incident.io app: it's easy to forget to archive incident channels and they can sit around for months after the incident is over. So we're happy to introduce auto-archiving as our newest feature! We'll automatically archive incident Slack channels for you after either 7, 14 or 28 days.

You can enable auto-archiving in the Settings section of your incident.io dashboard, in the "Automation" section.

Note: We can keep your Slack workspace tidy for you, but make no promises about your physical workspace.

What we shipped

Here's what else we've been up to this week:

• 🆕 You can now link an existing Jira ticket, Linear ticket or GitHub issue to an action or follow-up
• 🆕 Incident list CSV exports now include incident duration in seconds
• 💅 Auto-close now lives in the Automation section of Settings, next to Auto-archive
• 💅 When we auto-close, we'll auto-fill incident lead and summary if they're not set
• 💅 We can now show the content of pinned messages in the timeline after the incident channel has been archived
• 💅 Polished up the tutorial flow and improved some of the messages we send on install
• 🐛 Fixed removing a value in a multi-select dropdown, where it would remove all the selected values instead of just one
• 🐛 Fixed a bug where timestamps wouldn't clear when you remove them from incidents
• 🐛 Fixed the appearance of dialogs on mobile screens.

Happy New Year, from all of us at incident.io!

Winding down for Christmas is a great time to tackle feel-good product feature requests.

This post will focus on the changes we made to workflows, allowing you to:

• Enable and disable specific workflows
• Automate sending emails to key stakeholders
• Send SMS messages for urgent incidents
• Automatically assign incident roles

Let's dive in!

Enable and disable specific workflows

You can now temporarily disable specific workflows, making it easy to build and test a workflow without worrying it might run against on-going incidents.

To disable a workflow, press the "Disable" button at the top of the workflow page, then use the "Test" button to run against an incident of your choice.

When you're done, use "Enable" to reactivate the workflow.

You can use any incident for testing- we recommend creating a test incident (/inc test) exclusively for this purpose.

Automate sending emails to key stakeholders

Lots of companies have mailing lists that should be alerted when serious incidents occur (execs@, red-alert@) and for subsequent updates.

You can now build workflows to send these emails, using the editor to template incident data into the email.

Start with the Urgent incident template, or create your own with a send email step like:

Here's an email sent with this configuration:

Use this to subscribe executives or senior leadership to incident updates, or have teams hear about incidents of a specific type (ie, the Legal team for all Data breaches) by filtering on incident Custom Fields.

Send SMS messages for urgent incidents

People want to hear about urgent incidents immediately, not when they next check their email inbox.

For this, you can now create workflows that send an SMS message, including details about the incident or instructions.

Start with the Send SMS message template, or create your own with a send SMS step like:

Here's an SMS sent with this configuration:

If you're looking for a lightweight pager, this should do the trick! And if you want access to a workflow step that phones people, please let us know and we can enable it for you.

Automatically assign incident roles

Last but not least, you can now use workflows to automate assigning incident roles.

Depending on your company, you might want to:

• Automatically assign the incident lead to whoever first enters the channel, to ensure there is always a lead
• Set the Legal Counsel to Sally when the incident type is Data Breach, as she handles all data related legal work

Create a workflow with an "Assign incident roles" step, like this:

This would assign the person that reported the incident as the Incident Lead — this ensures you've always got someone leading the incident.

Round-up

That's it for workflow related changes we released over the break.

And as usual, here's a list of the smaller ticket items:

• 🆕 Expose incident reporter as a variable for incident related workflow triggers.
• 💅 Improved the after-first-incident message, helping people find the tutorial and giving better instructions to get started using us.
• 💅 Lots of work on the setup flow, from making error states clearer, to helping people reinstall the app when Slack scopes have changed.
• 🐛 Fix settings page bug with auto-close incidents where the duration would not be displayed when the page was refreshed.
• 🐛 Workflow editor would go wrong when using complex variables, creating § symbols- after a bit of painful debugging, this is now fixed.
• 🐛 Fix bug that prevented removing the auto-call link from settings.

From today, we're releasing an interactive tutorial that can teach people the basics of using incident.io, including:

• Providing incident updates (/inc update)
• Assigning roles (/inc lead)
• Creating and managing actions (/inc actions)
• Pinning content into the timeline (:pin:)
• Closing an incident (/inc close)

You can start the tutorial by typing /inc tutorial in any of your Slack channels:

Completing the tutorial takes about 5 minutes, quick enough to do just before joining an incident, or during a coffee break. Becoming more familiar with incident.io means you improve how you respond to real incidents- definitely worth a few minutes to try!

Just as responders can benefit from familiarity with the Slack app, it's important to understand the incident homepage, and how actions in the incident channel are reflected to anyone keeping track via the website.

For that reason, the tutorial ends by encouraging people to visit their incident page, where we'll trigger an interactive tour that explains each section:

Next time someone asks "how do I use incident.io?", give them /inc tutorial for a personalised walkthrough.

This is our last official changelog of 2021, but stay tuned for some feel-good content to enjoy during your Christmas break!

What we shipped

• 🆕 Create tutorials via the /inc tutorial
• 🆕 Product tour of the incident page when sent by the tutorial
• 🆕 We're officially SOC 2 certified! Read the blog post here.
• 💅 Use a consistent tone of voice across the product (we, not I, or the cat's mother).
• 💅 Cache issue tracker form options, such as the Jira project, the GitHub repo, or Clubhouse user- this should make exporting of actions much nicer.
• 💅 Lots of work on our marketing page to ensure sign-ups are easy, and we direct people down the appropriate funnel.
• 💅 For the /inc update flow, auto-fill the incident summary from the update if not set, and provide the last update for reference when writing the summary.
• 💅 Default to creating follow-up actions from the Slack app when an incident has been closed.
• 🐛 Fix a bug where incidents page loads indefinitely when you have no incidents.

Now you can customise your workflows even further! We've supercharged workflow message steps with a rich text editor, allowing you to format your messages easily, add links, and - most excitingly - add special variables into your messages.

Choose from a whole host of context-dependent variables, including information about the current incident, message and/or user (depending on the trigger you’ve selected).

You can also use Slack's mrkdwn syntax to take advantage of all the formatting options that Slack provides.

If there are any variables we haven't included that you'd like to see, let us know over at the Community!

What we shipped

• 🆕 Messages sent by workflows can now contain special context-specific variables, like the name of the incident.
• 💅 We've added a new Workflows template to send a message when a post-mortem is added to an incident.
• 💅 You can now create a workflow triggered by an incident update being shared.
• 💅 We've given the app setup flow a bit of a makeover.
• 💅 When we fetch all your incidents to show in a list, we now do it in batches which will speed up loading times
• 💅 GitHub action exports now support labels!
• 🐛 We fixed some missing help text on the "Change to follow-up/action" button in incidents.

After many weeks of work, we're delighted to announce the latest feature of the incident.io platform: Workflows. Configure your processes once, and we'll make sure you follow them, every time ✨.

Workflows allow you to automate certain actions and behaviours based on specific triggers. We provide a set of building blocks which you can put together to encode your organisation's processes into incident.io.

That means you can now:

• Automatically page the right person depending on the incident type.
• Build runbooks by creating lists of actions at the start of an incident.
• Prompt your Incident Leads to go through a Decision Flow if they mention a potential data breach.

Check out our blog post for more details on what Workflows can do, or jump straight to the app to get started.

What we shipped

• 🆕 You can now encode your processes into incident.io using Workflows.
• 💅 You can now filter all dropdowns in the app by starting to type in them, including when exporting actions to an issue tracker (thanks for the nudge on this, Ubie).
• 💅 We now show user avatars when exporting issues to Jira, Github or Linear.
• 💅 You can now see both full names and usernames when exporting an action to GitHub.
• 💅 We've tidied up the settings navigation so it's easier to find what you're looking for.
• 💅 We've added an RSS feed to our blog so you can subscribe here.
• 💅 You can now search for an incident by its ID (e.g. by searching 139 you'll get INC-139).
• 💅 We no longer show all participants on the incident homepage by default: we'll just show a few and allow you to see the rest by clicking 'Show more'.
• 🐛 You can now unset a call link from Slack (it didn't used to allow you to, sorry folks.)
• 🐛 Links to external issue trackers will now open in a new tab.

If you use Zoom, check out our docs on how to get setup in your incident.io workspace.

Our list of integrations grows larger this week with the addition of Zoom, the video conferencing product used by a number of our customers.

Installing Zoom will enable the "Automatically create incident call" settings in the dashboard. If configured, incident.io will create a Zoom meeting whenever an incident is declared, which reponders can use for high-bandwidth discussion.

Creating a call link has always been something we advocate for at incident.io, especially in an increasingly remote workplace. We're excited to get this released, and look forward to increasing the depth of the integration now our app is officially part of the Zoom Marketplace.

We're awaiting review for our application to the Google Marketplace (for Google Meet), and expect to see first-class support for video-conferencing in our upcoming workflow product.

What we shipped

Most of our focus was on building the first version of workflows, which we managed to get to a working prototype! 🎉

But life can't always be about workflows. Here's a few of the changes we snacked on alongside our main:

• 🆕 Support for auto-creating Zoom meetings
• 🆕 Wrote a load of blog content which we'll be releasing over the next few weeks — read our post on how we feature flag or check out our observability setup as content from last week!
• 🐛 Fixed an issue with our issue tracker clients where requests weren't being retried — this should improve reliability when exporting actions to external issue trackers
• 🐛 Lots of small UI tweaks and fixes. We're proud to say all buttons are now (and should always have been, if we're being honest) the same size...
• 🐛 Jira auto-export was subtly broken when some fields were left unset (thank you TrueLayer for your patience!)
• 🐛 Updating a call link in Slack, if you did this with a non-URL, used to break- sorry about that!
• 🔜 A whole lot of work on workflows, coming soon!

You can now close incidents from the dashboard (instead of having to be in the relevant incident channel). This is particularly useful when tidying up old incidents ready for analysis. It also continues to bridge the gap between what's possible from inside Slack vs. the dashboard.

🐍 Wiggle Week

When we're working on a big feature like announcement rules (and the workflows engine which powers it), we work hard to stay focussed on the task at hand. This can make us feel uncomfortable as we constantly come across things that we'd like to improve: either from customer feedback or from our own usage of the product.

To make sure we stay focussed on the big ticket items, but not letting our backlog of polish and bug fixes grow uncontrollably, we schedule time between big projects to knock items off the list. We call these wiggle weeks (an idea inspired by a great Intercom blog post). As you've probably guessed, this was one of those weeks.

What we shipped

• 🆕 You can now close incidents from the dashboard. Thanks for the nudge on this one, Render.
• 💅 If we need you to re-install the app to unlock some shiny new features, we'll tell you in the dashboard using a shiny new banner.
• 💅 We've tidied up the UI where we display actions, to make them work a bit better on smaller screens, or when you have multiple issue trackers enabled.
• 💅 You can now edit and export follow ups from the follow-ups page (instead of having to view the incident)!
• 🐛 Editing actions in the web UI will now refresh the incident timeline, so you will see the most up-to-date information.
• 🐛 Our GitHub integration didn't used to work well if you had lots of repos or lots of users. It should now work much better! Thanks for letting us know Sourcegraph.
• 👷 We've now got better tools to debug customer issues faster, so when you do report that something's wrong we can jump on as fast as possible.

Starting today, you can use the incident.io dashboard to create Announcement Rules, allowing you to subscribe a collection of Slack channels to announcements for incidents matching specific conditions.

What are announcements?

incident.io has always shipped with a default #incidents announcement channel, which receives a post whenever an incident is created.

While our users often say how much they love this channel, with the increase in visibility it brings, not everyone wants to hear about every incident.

Perhaps your exec team only want to hear about incidents where the severity is greater than Major?

Or maybe Compliance need notifying whenever an incident of type Data Breach has been created?

Announcement Rules

That's where Announcement Rules come in.

Up until now, we'd announce each one of your created incidents in one channel - #incidents. Now, you can choose additional channels to post certain types of incident into - and define exactly what those criteria are.

From the incident.io dashboard, create rules that match against incident dimensions such as:

• Incident severity is greater than Major
• Incident status is either Investigating or Fixed
• Custom field Affected Teams includes Data

Whenever incidents are created that match these conditions, we'll send an announcement post to the channels configured on the rule.

Once the rule is created, we'll notify the channels involved to let them know they've been subscribed.

And as a final sweetener, an often requested feature is the ability to change the default #incidents announcement channel.

This behaviour is now controlled by the default announcement rule, which - just like other rules - you can configure from the settings dashboard:

That's all folks! It's worth noting that announcement rules are the first of a few features powered by our underlying workflows engine, which we're hoping to expose as a first class product feature over the next month.

Watch this space!

What we shipped

And in the typical weekly round-up:

• 🆕 You can decide to announce certain incidents in different channels, depending on properties of the incident.
• 💅 You can change your default announcement channel in the announcement settings page.

We've been working hard to stay focussed so the team has been deep in announcements, but we've picked up a few other bits along the way:

• 🐛 We fixed a bug where you couldn't unassign an action from the dashboard. Sorry about that!
• 🐛 When you update custom fields in the dashboard, we'll now refresh the page to show your changes straight away.
• 👷 We've started using LaunchDarkly so we can release changes in a more controlled way, and so far we are loving it!
• 👷 We're looking to expand our team, and have launched a new jobs page with a few open roles. Watch this space for more 🔜.
• 👷 We are working to make our dashboard more accessible, and have made a few small tweaks to make incident.io work for everyone.
• 👷 Katie joined the team as our first non-engineering hire! She's going to be working on a wide range of stuff from strategy to SOC2, and she'll introduce herself soon.

This week we've made it even easier to filter your incidents on the web! Maybe you want to view only the incidents assigned to you as incident lead, or show any incidents with a severity of Major or greater. Or BOTH. So many filtering possibilities!

Where previously you could choose to show incidents from the last 7 days, 30 days or 90 days, we've added some more complex date filtering as well, including showing all incidents within a specific date range.

You can even copy or bookmark the URL of the filtered view to share it with others.

What we shipped

• 💅 We introduced some new and powerful incident filters, improved date filtering, and added filtering by assigned roles.

• 💅 You can now disconnect integrations from the Settings area!

• 💅 Incident postmortem docs now have a link to the incident's Slack channel

• 💅 We now show a warning when you're updating your public status page from within Slack, to make sure it's what you absolutely want to do!

• 💅 We've now got a shiny new Pricing page!

• 🐛 Changing custom fields on the incident detail page will now update the UI afterwards.

• 🐛 We fixed a bug where the Auto Close Incidents setting wasn't loading any content.

• 👷 Sophie joined the team as our newest engineer! She'll introduce herself soon - for now she's been picking up some of the things you've been reporting in the incident.io Community.

This week we bring you a brand new way to declare incidents from the dashboard, as well as a whole host of improvements, polish and bugfixes!

We also recently announced our new $4.7m funding round, led by Index Ventures and Point 9, and moved to a new office to support our rapidly growing team. It's been a busy few weeks, and we're currently shipping product surrounded by cardboard boxes and cables we can't quite place, but we're not slowing down!

We've been asked a number of times whether it's possible to declare an incident from the dashboard, to which the answer has been "not yet", until now! The eagle eyed amongst you will notice we've recently introduced the ability to declare incidents from the dashboard.

We've also registered a neat little domain - inc.new to make it even easier to do it, just type inc.new into your browser, and it'll take you straight to the "declare an incident" screen - give it a try!

As always, we're excited to hear what you think about what we're shipping, and we rely on your feedback to shape what we build next - let us know your thoughts in the incident.io community Slack!

What we shipped

• 💅 We now say thanks when you invite us to an incident announcement channel, always good to be polite!
• 💅 We gave our website a small spruce-up - got to look our best for all our lovely new customers!
• 💅 We're now able to post in (not read) any channel. In some cases we were still sending direct messages to get around some auth scope issues, but we now reply to you in-channel if we can. If you're not seeing this, you may need to re-install the app, just let us know and we can help.
• 💅 For customers with multiple status pages configured, we now help you set them up without any need to speak to us. We love hearing from customers but this felt like a support call nobody should need to make!
• 💅 Messages which were pinned and then deleted were showing up as "experiencing an error" in the dashboard. Whilst technically true, it didn't feel very useful. Now we hide them.
• 🐛 For some customers, exporting the postmortem document to Google docs resulted in the timeline table only taking up half the width of a page. Fixed! Thanks to Peter @ BottlePay for raising it.
• 🐛 Follow-up actions that were closed and then re-opened, weren't appearing on the follow-ups page. Now they are! Thanks to Girish at DBT labs for reporting that one!
• 🐛 We were seeing some very rare, very strange behaviour where after closing an incident, it'd magically and silently re-open itself. Unacceptable, but after quite a gnarly investigation involving ORM behaviour, race conditions and a lot of quizzical facial expressions we tracked it down. Thanks to Michael @ Bud for flagging it!
• 🐛 We were getting reports that the bot felt a lot chattier than normal, which didn't seem right - we try hard to avoid emulating the microsoft paperclip of old and have several safeguards in place to try and avoid spamming incident channels at the wrong moment. It turned out we'd subtly changed some of the behaviour, now fixed. Thanks to everyone who brought it to our attention
• 🐛 CSV files now use UTC for their timestamps. Timezones are something that multiple customers have raised and we're thinking about how to best represent them in the app more generally, but for now, aligning CSVs with other parts of the app felt sensible.
• 👷 We've invested engineering time in massively improving our logging and exceptions, solving a few weird bugs where sourcemaps weren't appearing as expected. Now they're so good you almost want to introduce bugs...
• 👷 We've refactored a subset of the code that powers our API, laying the foundations for a lot more product work we have planned in the near future, and helping us ship changes even faster

This week we're launching an early preview of a feature we're calling Decision Flows, which aims to make routine decision making easier and more transparent during incidents.

It all started two weeks ago, when a late night YouTube binge led me to videos about GDPR (welcome to my life!) and it reminded me of an interaction I had with the DPO at a previous company. They wanted engineers to assess whether an incident might need their involvement, and to help them decide they shared a document with a list of criteria. It seemed reasonable, and in isolation it might be doable, but we had multiple similar requests from different corners of the company: marketing asking to be looped in when the customer impact ticked certain boxes, Risk and Compliance sharing criteria for regulatory reporting, specific guidance for when to put up the public status page – the list goes on.

Soon, you have folks needing to consult 13 different documents to make the right decisions, and they're so focused on doing the right thing they forget they have an incident to resolve.

This got me thinking: what if instead of jumping out of Slack to follow a doc, you could be asked a sequence of questions to help make the right decision? A few hours later, I had a scrappy prototype built in Slack to see how it felt...

Two weeks later and we have a way to configure these Decision flows in the dashboard and a `/incident decision` command to run through them in any incident channel.

We're excited to see what you do with them - let us know in the incident.io community Slack!

What we shipped

There's work in the pipeline that we're excited to share soon, but for now here's some of the other things we've shipped:

• 💅 We often get asked to mark incidents as a 'test' after they're declared by mistake. You can now do this yourself by hitting the 'Mark as test incident' incident button on the incident dashboard.
• 💅 You can now filter to include or exclude test incidents when searching and exporting your incidents in the dashboard.
• 💅 Ever written something in a Slack channel that you wanted to convert in to an action? You can now create actions just by responding with a 💥.
• 💅 If you're using the Jira auto-export feature, we now link to your Jira ticket from the announcement post in #incidents.
• 🐛 Some folks were unable to access the incident Slack channel by clicking the 'Join Incident' button in #incidents. We now link to the channel using a native Slack link so this shouldn't be a problem.
• 🐛 We found and fixed an issue that meant durations were sometimes displayed incorrectly in the dashboard.
• 🐛 We fixed an issue that could lead to multiple public status pages being created. Fortunately, we were the only ones bitten by this.
• 👷 We've been changing how our frontend and backend talk to each other, whilst paving the way for our public API.
• 👷 We've made a heap of changes to our logging and tracing infrastructure, which means we have beautifully structured logs and traces in the same place, and nicely linked to errors in Sentry. Debugging's never been so good.
• 👷 We've sped up our build pipelines, meaning we can ship code to production in under 5 minutes ⚡

This week we're doubling down on Jira; helping you connect incident.io to your existing processes.

Auto-export incidents to Jira

We know that a lot of our users use Jira, for a wide variety of use-cases surrounding incidents. We want to make it really easy to integrate incident.io with your existing processes, so that its simple and safe to start using incident.io. It also means you can keep using all your existing reporting processes without having to start all over again.

You can now configure your incidents to create a ticket in Jira whenever an incident reaches a pre-defined severity, by editing your automation settings.

The ticket will have a link back to the incident.io homepage so you can easily move between the two:

P.S. We want to make sure you don't get an avalanche of Jira tickets, so test incidents will never create Jira tickets.

What we shipped

We've spent a lot of the week working on our internal debugging tooling, so we can identify and fix issues quickly. We've also worked on a few things you might notice:

• 💅 We now support freetext fields when exporting to Jira

• 💅 The Jira export modal now walks you through selecting a project and then an issue type, so it's easy to see what fields you need to fill in.

• 🐛 If you pin a message that's in a thread, it'll now show up in the timeline in the dashboard and Postmortem

• 🐛 If you pin a message from google hangouts, that will now get added to the timeline

• 👷 We made our error reporting, stack traces and logs much more useful internally, to help us fix any issues that crop up even faster!

• 👷 We invested some time into optimising our build and release pipeline, we can now ship changes to you in half the time 🚀

With last week being just four days, we decided to tackle some small quality-of-life changes we know people have been dying to see.

Upgraded Jira ticketing

After our recent introduction of follow-up actions as a mechanism for tracking post-incident tasks, we've seen an uptick in people exporting their actions into external issue trackers.

If you're a Jira power-user, you may have struggled to export your actions if the project or issue type you were exporting into required custom fields.

After digging into Jira's API and getting to grips with the various different field types and auto-completes, we now support custom fields of whatever form you may have configured, all inside the incident.io dashboard:

Exporting actions into backlog issues is something we do ourselves at incident.io, pushing our post-incident tasks into Linear. We keep track of the link between your incident action and the external issue, and sync action states/assignees so incident.io reflects the current state of the issue.

Doing this allows us to provide you insight on how well you're actioning your incident follow-ups. View a summary of outstanding follow-ups in your dashboard or export to a CSV if you want to crunch the numbers yourself.

We'll be looking to do more with this data in future. Watch this space!

Initial incident message and invites

Loads of our users want to hook into incident creation to and customise the messaging given to responders, perhaps offering guidance or advice that might be useful to help get started.

As of this week, you can configure your incidents to:

• Post a custom message whenever the channel is created
• Invite specific people into the incident channel on creation

You can control the message and invites in your settings page:

That config would create incidents like this:

We want incident.io to be fully adaptable for your organisation's processes, as everyone has different wants and needs when it comes to running incidents.

These two features are a preview of a more powerful and general concept we want to build into the product, something we're calling workflows. We expect to roll these settings into workflows once it's ready, so watch this space!

We've updated our roadmap!

If you want to follow this work or see what we plan on hitting next, we've updated our product roadmap with our most recent plans.

Please send us any feedback or suggestions of what you'd like to see us build in the Slack community- we always want to hear from our users!

What we shipped

As always, a quick summary of what we released:

• 💅 Rejig order of incident creation messages for improved readability

• 💅 Speed up our incidents endpoint, helping the dashboard load faster

• 💅 Updated the public roadmap with our most recent plans

• 👷🏽‍ Building out interviews for a variety of roles- checkout our jobs page, we're hiring!

• 🐛 Support post-mortem copy-to-clipboard in Safari

• 🐛 Clear the actions edit modal of unsaved changes

This week, our focus is all about getting data out of our product to help out with analysis, reporting and tracking. You can now export data to a CSV from the incidents and follow-ups page.

From the incidents page you can export a CSV of the incidents that you're currently looking at. It will include all the information you see on that page (status, severity, name, description) as well as custom fields and roles you have configured.

From the follow ups page you can export a CSV of all the follow up actions across your organisation. This will include information about the action (description, reporter, assignee) and the external issue tracker reference if you're using one. There's also information about the incident on each row to help you sort, filter, report and apply your spreadsheet wizardry!

What we shipped

• 🆕 We've added a new help page for custom fields.
• 💅 We've added an alias so you can use /incident fields as well as /incident field to update custom fields.
• 👷🏽‍ Reworked our API to make the incidents page load faster.
• 👷🏽‍ Invested in our issue tracker integrations to avoid UI bugs due to us hitting third party rate limits, and make actions feel really snappy in the dashboard.
• 👷🏽‍ Continue porting API endpoints to goa, allowing us to type-check our frontend code against our backend implementation.
• 🐛 The 'team' block was missing from the default post mortem template. We've put it back.
• 🐛 Mandatory roles are now only required when closing an incident (and the settings page makes that clear).
• 🐛 The current assignee now correctly pre-populates on the edit action modal in slack.
• 🐛 Timeline items showing pinned bot messages used to break the timeline in the dashboard. We fixed that.

incident.io provides actions, our lightweight toolkit for tracking and assigning work during incidents.

After speaking with our community and collecting feedback, we spent the last week making a number of improvements to actions that we hope will better fit your incident workflows.

What are actions?

We believe there are two types of action you create during an incident – those that need doing now, and those that should be followed-up after an incident has been closed.

Actions that need doing now might be:

• Reboot that server!
• Failover to that region!

While follow-ups could be:

• Improve test coverage of ActionsService
• Degrade gracefully when Redis is down

We're shipping follow-ups

You can now mark actions as follow-ups, helping separate work that needs doing now from tasks that can be handled post-incident.

Create follow-ups in Slack like any other action (/inc action Thing) but with "When does it need to be done?" set to "After the incident":

Sometimes you'll create an action, then realise it's best handled as a follow-up. Moving actions into follow-ups (and out!) can be done from the /inc actions modal:

Once created, follow-ups can be managed in your incident dashboard, where they are shown separately from your other incident actions:

We expect people will review their actions as part of closing an incident, taking the opportunity to promote an action to a follow-up if it's still relevant.

If this is how you use them, we suggest exporting any follow-ups to your issue tracker (see the "Export to Linear" button) to link against your ticket, ensuring incident.io will sync the follow-up to your issue tracker.

Following this workflow will mean you can use the new "Actions" view to track outstanding follow-ups across all your incidents, giving you a picture of how your organisation is doing on their post-incident work:

That's not all!

While follow-ups are this weeks big news, we took the opportunity to improve actions more generally, too.

Expect some major quality-of-life improvements when working with actions from within Slack, such as:

• /inc actions defaults to showing outstanding actions only, assuming responders will want to clearly see what remains (rather than what they've already done!)
• Made tough decisions of which buttons to put in our Slack overflow menus- with a maximum of five quick-links, we've tried picking the most relevant buttons for the specific action
• Create an 'edit action' modal for making several changes at once

See it in action here:

What we shipped

Lots! Mostly covered above, but a few small tweaks

• 🆕 Follow-up actions to track work that happens after closing an incident

• 🆕 New actions tab for viewing outstanding follow-ups across all your incidents

• 🆕 Mark actions as not-doing, helping responders focus on what they need to do now

• 💅 Simplify the /inc actions modal and clearly separate actions by status (Slack)

• 💅 Provide an action edit modal for changing action details (Slack)

• 💅 Show quick-access buttons to transition actions between states (Slack)

• 💅 Separate follow-ups from in-incident actions (dashboard)

• 💅 Use consistent action status names (Open, Outstanding -> Open, etc)

• 💅 Filter incidents by "has outstanding actions" or "has post-mortem" (dashboard)

• 💅 Now display severity descriptions in the incident create modal (used to just be in the update modal, now its in both) - thanks Tiago!

• 💅 Bookmark your favourite incident filters, so you can share your search queries

• 💅 Custom fields now appear in the incident recap view

• 👷🏽‍ Continue porting API endpoints to goa, allowing us to type-check our frontend code against our backend implementation

• 🐛 Clear create action modal inputs after successful create (dashboard)

• 🐛 Notify incident channel whenever actions change in the dashboard

• 🐛 Fixed attribution of timeline items when you pin another persons message

We've added filtering so you can more easily find the incidents you're interested in.

On the incidents page, we've still got the free text search, but have added some more filtering options. This enables you to look at incidents with particular characteristics; either to find a particular incident or to help do some high level analysis. We also think this might be useful for running regular incident reviews and for anyone wanting an 'at-a-glance' view of the activity over the last week or month.

Currently you can filter by:

• Date — when did the incident happen?
• Severity — what severity did you assign to the incident?
• Status — what is the current status of the incident?
• Participant(s) — who was involved in the incident?
• Any custom fields that you've defined

There'll be more filtering options coming soon - please let us know what you'd like to see next!

What we shipped

• 🆕 You can now pin a message in an incident channel by reacting to it with various pin-like emojis (📌, 📍, 🔖, 💭, ⭐)
• 💅 We've given the 'update roles' modal a new look so it's a bit easier to use.
• 👷🏽‍♀️ We spent some time working on improving some fundamentals under the hood, particularly around our API.
• 🐛 When updating incident roles via /incident roles, mandatory roles are now actually mandatory.
• 🐛 We can now gracefully handle it if someone tries to use the same option value twice when adding a custom field.
• 🐛 The insights section of the dashboard will now recalculate when you open or close an incident, so you won't see out of date information!

P.S. I really wanted to call this blog post ge-filter fish, but we decided that too few people would understand the reference. This post-script was the compromise.

incident.io ships with a set of commonly used fields for an incident, such as the severity and who's leading. However, a feature that virtually all of our customers have been asking for is the ability for them to add their own custom fields to incidents. They want to be able to track things that are specific to their business.

We're really excited to release Custom Fields to do exactly this. Custom Fields are a way for you to define the data that you want to collect on your incidents, and have incident.io collect it for you before, during or after an incident.

Here's a few examples of useful Custom Fields that we commonly see:

• System Affected, which would let us collect which systems were affected by a particular incident.
• Type, to reflect whether this is a Platform Incident, Security Incident, or HR Incident.
• Team, which denotes which team is primarily responsible for response, and follow-up.

For defining what data you collect, we currently support:

• Select: selecting a single value from a drop-down list
• Multi-select: selecting multiple values from a drop-down list

We'll shortly support collecting text values, number, booleans and dates.

We're really excited to have shipped this. It's another step towards incident.io becoming the single source of truth for all of your incident data within your organisation. We can't wait to see what you build with it.

To get started, head over to your Settings page and add your first Custom Field!

What we shipped

• 🆕 We've built a new Jobs page. If you like what you see here, then we'd love to talk to you!
• 💅 We've added an RSS feed link for our Blog. Thanks to SRE Weekly for the prod here!
• 👷🏽‍♀️ We've moved all of our infrastructure into Spacelift to make it far easier to manage. We were up and running in a day, and we're big fans so far. Check them out!
• 👷🏽‍♀️ We've hired Lisa and Lawrence, who will introduce themselves soon. As part of this, Lawrence put a lot of effort into making it as easy as possible to get started as an engineer. Lisa then used it the following week and got up and running in just a few hours.
• 👷🏽‍♀️ We used to allow you to carry on creating incidents indefinitely past your trial period. We now allow you to create a maximum of 3 incidents after your trial period before we ask you to become a paying customer. It's only fair, we think!
• 👷🏽‍♀️ We've made some huge improvements behind the scenes to our Jira integration to make it more flexible, and more powerful. This will allow us to power lots of exciting things in the future — more information to come!
• 🐛 Due to our strict Content Security Policy, some of our onboarding Loom videos were not being shown. We fixed that.

On top of the usual collection product improvements, this week we've also shipped our new blog. The old one felt a little uninspiring, and since we've got lots we want to share, we thought a little overhaul was a good investment in time. To mark the occasion we shared the first in our interview series, this time featuring Colm Doyle from Slack.

We've had a few things competing for our time over the last few weeks, which has slowed the pace of change a little. Next week's looking pretty clear though, with plenty of time to devote to product and engineering. Stay tuned!

What we shipped

• 🆕 We launched our lovely new blog and the first in our new incident interview series, this time with Colm Doyle from Slack. Kudos to Pete here!
• 🆕 We've made some changes to make incident summaries feel more connected to the incident. After you /incident update, we now prompt you to check that the overall summary still makes sense. A well written summary can go a long way to keep folks on the same page.
• 👷🏽‍♀️ We've been working with an external company to carry our a penetration test against our app and infrastructure. We're confident in our security setup, but it's great having someone try to find any holes. So far, things are looking great 🔒
• 👷🏽‍♀️ We've added some debugging tools to our codebase to help us find and fix things like memory leaks in future. As they say, failure to prepare, is preparing to fail.
• 👷🏽‍♀️ We used to perform a round trip to Slack on one of our identity APIs, which meant we coupled the success of that call to Slack. Given we were getting data that's unlikely to change often, we've moved it into cache.
• 💅 We used to nudge you to set the 'incident lead' even if you'd changed what the role was called. Now, if you call it a commander, we'll call it a commander 🤝
• 💅 We've shorted the timestamp we use in postmortems. Nobody needs millisecond precision here.
• 💅 We used to nudge you about the incident being quiet, even if you were in a monitoring state. When you're monitoring it's not uncommon for the channel to go a little quiet so this wasn't particularly helpful. Longer term, we'll make all of our nudges customisable, but for now we've restricted this nudge to investigating and fixing stages of the incident when we expect activity in a channel.
• 💅 We no longer show inactive or removed users in dropdowns, since assigning an action to someone who longer works for you probably isn't a wise move.
• 🐛 We fixed a bug that occasionally meant images wouldn't show up in the incident timeline. Long story short, when you loaded the incident homepage we'd generated a url for the image which would be valid for 10 minutes, and then cached the linked timeline item for 15 minutes. In those last 5 minutes, the image url had expired, but we still served the timeline with it. All fixed now though, so pin away!
• 🐛 Postmortem exports weren't working on Safari, but now they are!
• 🐛 We fixed a few minor issues with our Jira integration, and have some bigger changes on the way soon.
• 🐛 If you reopened an incident, there was a chance the incident homepage would stop working. Not anymore though! Thanks to RD Station for their help getting to the bottom of this 🙏

Over the last week we've been busy making it even easier for your whole organisation to onboard with incident.io. Whether it's simple step-by-step instructions for the installer, or helpful messages to folks after they've participated in their first incident, we've got you covered.

What we shipped

• 🆕 We've overhauled our sign-up and installation process. You'll now be taken through a full onboarding process in preparation for your first incident.
• 💅 We now handle redirects during login, so when you click that link to an incident, you'll actually get there.
• 💅 After providing an update with /inc update, we'll now prompt you to check the incident summary still makes sense. A few minutes getting everyone on the same page can save hours later on.
• 💅 It's possible to close a Statuspage.io incident without marking your components as operational. We now smooth that rough edge by prompting you to resolve them all together.
• 👷🏽‍♀️ We shared a blog post about a Postgres feature that caused some incident numbers to jump by 32. If edge cases in databases are your bag then check it out!
• 👷🏽‍♀️ We got caught out by a Slack workspace where channel creation was restricted to admins. We now handle that case to avoid half-complete installs.
• 👷🏽‍♀️ We've made improvements to our logging setup, so it's easier to pinpoint bugs when they (very rarely 😉) pop up.
• 🐛 We've fixed a problem that would sometimes prevent pinned Slack messages with images from showing up in the timeline. All fun and games until caching gets involved, but we've fixed this now.
• 🐛 When you closed an incident via the /inc update command whilst also changing the severity, we'd show the old severity in the closure modal. Now, thanks to a report from the folks at Ziglu, we show the correct one.
• 👷🏽‍♀️ We used to restrict Slack channel names to simple characters and numbers, despite Slack supporting diacritics. We now support everything Slack does, which is àwęšõmê!

Instead of doing just one thing last week, we made lots of smaller improvements to the product. We also got a lot of internal admin off our plate to keep the paperwork gods happy and keep the wheels turning.

We won't bore you with the paperwork, but we hope you enjoy the new features and improvements!

What we shipped

• 🆕✨ Better incident declarations. We now show you a 'jump off' modal after you declare an incident (pictured). Might just be us, but we were finding it hard to quickly find incident channels after declaring an incident. This makes it a lot easier.
• 🆕✨ Automated incident closure. To keep your data nice and tidy, we'll automatically close incidents for you that haven't seen any activity for a time period you specify. Enable it in your settings to try it out!
• 🆕✨ Nudges to update your Statuspage. If you've created a Statuspage incident for your incident, but haven't updated it for 30 minutes or more, we'll send you a nudge to remember to do that. Now it's easier to keep your customers in the loop.
• 💅 We've unified the severity, status and update commands behind the update command. Fewer commands to remember makes the product simpler. However, using /incident status will still work just fine.
• 💅 We added support for Microsoft Teams as a video conferencing provider. If you paste a Teams link into an incident channel, we'll now prompt you to to add it as the call link.
• 👷🏽‍♀️ We added a vulnerability disclosure policy to our website. If you spot any security issues, please follow the instructions provided there.
• 👷🏽‍♀️ We made a number of resiliency improvements, mostly focussed on our job processing infrastructure and our statuspage integration. Hopefully nothing too noticable, but it helps our team sleep better at night knowing it's all shipshape.
• 🐛 Organisations with 5 or more severities would have all of their severities appear in a delightful mauve on the Insights graph. Whilst that was quite nice, it wasn't intended, so we fixed it 💜 (thanks to Duffel for reporting!)
• 🐛 The dates in our insight graphs were sometimes appearing offset by a month. All fixed, no more time travelling (thanks to Duffel!)

Incidents are a team sport, and to help everyone get involved we built incident actions. With a quick /inc action in Slack, anyone can view, create, and pick up actions, making it easy for folks to self organise around whatever needs to be done to get things back on track.

We got some feedback that the actions interface in Slack could be a little clearer though, for example the layout was a little cluttered and it wasn't easy to see who created what. We've now refined the design and tweaked what we display so it's even easier to get a clear picture of what's going on.

Got thoughts on this or any other part of incident.io? Jump into our community Slack and let us know!

What we shipped

• 🆕 We want the trial experience to be super low friction, which is why we don't ask for a credit card up front. To save us personally nudging folks at the end of their trial, we now show you when you're close to the end, or over your trial period.
• 💅 The actions modal has been given a little overhaul. We've simplified the layout so it's easier to see what's going on, and we now show who created open items so you can ask them for more details if needed.
• 💅 If you typed /incident help from outside an incident channel, we used to open the create incident modal with an incident title of "help". You were probably after some actual help, so we now show you modal with some tips. Thanks 100/x for this feedback!
• 💅 When you joined an in-progress incident, we used prepend the summary with "All systems go in here!". On reflection, this wasn't the clearest language when your systems often weren't going. We're a little less ambiguous now.
• 💅 We now support typeahead search in the Escalations modal. You can now filter to find the person or escalation policy you're after.
• 💅 We used to show PagerDuty users and escalation policies in a non-deterministic order. That wasn't especially nice, so we now show them in a consistent order.
• 💅 We new prompt you to provide an incident update if you haven't done so for a while. Keeping folks on the same page is important, and we know how easy it is to forget.
• 💅 We've spruced up our login flow, so it's no longer just a lonely login button on a blank screen(!). It's had some design applied, and we explain what people in your organisation will be asked for and why.
• Links to the incident homepage now automatically embed the teamID wherever we can, so logging in with slack auto-fills to the right workspace. This helps people avoid accidentally signing into the wrong one, something we've seen bite a few of you!
• 👷🏽‍♀️ We've improved how we poll PagerDuty for updates. This means we need to do less work, and we're also a little less trigger-happy with the PagerDuty API. It's nice to be considerate.
• 👷🏽‍♀️ We've made some reliability improvements in the way we run concurrent code in the app. We'll spare you the nerdy details, but things are much more resilient as a result.
• 🐛 If you had more than 100 components configured in Statuspage.io, we'd fail to render them in our Slack modal. All sorted now; create components to your heart's content!
• 🐛 If you pinned a Slack message from a bot, we'd fail to put add it to the incident timeline. All sorted now - pin away!
• 🐛 We use content security policies in our front end, to keep things safe and secure. We were being slightly too secure with our shiny new Intercom chat, which meant some functionality was broken. We've fixed that - feel free to give it a try and say hello 👋

We're fanatical about great support here about incident.io. Every customer gets a shared Slack channel to make it easy to ask questions and give feedback. We've now made that even easier by adding Intercom to incident.io.

Whether it's a question about how to add a custom incident role, or feedback on an improvement we could make, you can now send that to us in a couple of clicks. Click the button in the bottom right hand corner of incident.io to get started.

Behind the curtain, Chris, Pete and Stephen will be diligently answering your questions, fielding your feedback and blushing at your compliments. Looking forward to hearing from you!

What we shipped

• 🆕 We've added two new Looms to help you learn how to set up and use incident.io. We send now send you a link to a video tutorial when you install incident.io, and when you create your first incident. We love you, Loom!
• 💅 When you add an update via /incident update, we now default your next update to 30 minutes, as part of encouraging better defaults.
• 💅 We've sped up every web request by approximately 200ms by introducing caches in the right places. 🚀
• 💅 When you join the channel for a closed incident, we'll send you a message with some useful links to the homepage, and to your debrief.
• 👷🏽‍♀️ We spent a lot of last week working on fundamentals to make our application scale better. We can't give you all the juicy details, but we promise we're working hard!
• 🐛 After releasing incident updates last week, we discovered that we were still reminding you for updates after the incident was closed. That was silly, and we don't do that anymore.
• 🐛 We send you a nice message to get you up to speed when you join a live incident channel. In some circumstances, we'd send the summary twice as part of that. Regardless of how beautifully written your summaries are, we only want one copy to appear.

Good incident response is about good communication. In incidents, it's particularly helpful to provide a small, structured update to the organisation on a regular basis. This keeps everyone on the same page, and helps resolve worries about when the next update will be coming, too.

We're always looking for ways to make good communication easier, and a few customers have mentioned that using seperate commands to udpate the severity, status and summary often felt a bit harder than it needed to be. You might not remember to update the status, or forget to update the summary after new information came to light — we certainly did!

We think all these fields work better together, so last week we set to work on incident updates!.

Updates are an easy way to provide a small, structured update to the rest of the organisation on a regular basis. We tie together existing fields like Status and Severity, with a new field allowing you to share any other context. Additionally, if you set a reminder for when you're expecting to send the next one, we'll remind you at just the right time ⏰

Hopefully this makes internal communications a lot easier, and reduces some of the burden when it comes to communicating regularly. Give updates a spin in your next incident, and let us know what you think!

What we shipped

• 🆕 We added grouping to the /incident command, to make it easier to understand what you need to do in the middle of an incident. You can now Communicate, Coordinate, Review and Help to your heart's content.
• 💅 We changed the text when asking for feedback to make it clear that the feedback is for your organisation to improve at incident response, not feedback about incident.io. As much as we would love that feedback, we think it's more important for you to share it amongst your team.
• 💅 Adding a postmortem document from the web will now update the incident channel.
• 💅 Statuspage updates are now asynchronous when you post them. This means that we're more resilient to failure if Statuspage is down, or a little slow.
• 👷🏽‍♀️ We spent a bunch of time improving the reliability and correctness of our Statuspage integration. It's now much faster, and more failure-tolerant. Thanks for the feedback, Render!
• 👷🏽‍♀️ Announcing and updating the incident in your #incidents channel now happens asynchronously. This allows us to be more resilient to failure, and provides a 300ms speed up for every command 🚀
• 🐛 When you created an incident from a message, we weren't attaching the name of the channel you created it from in our response to you in certain circumstances. That was silly, and we're sorry. It's now fixed.

It's a been a while since we last posted one of these, and despite the radio silence we've been busy behind the scenes at incident.io HQ. Firstly, we now actually have a HQ, and it's in an old firestation - very fitting. Pete and Stephen have left their full-time jobs and are now working 100% on incident.io.

This week we've decamped to an eco-lodge in the Wye Valley to plan for the next few months. We've got hundreds of ideas and a mountain of valuable feedback, so we're taking a week to refine it all into something amazing.

No more breaks now. Expect weekly changelogs to be weekly. All systems go.

Thanks for the feedback!

We received some lovely feedback from customers over the past week — this kind of thing makes our day, thanks folks!

We at @Farewill started working with the fantastic @incident_io team earlier this year, and honestly it makes life SO much easier to have seamless tooling when everything is on fire. Watching our engineers working in the incident channel to check how we're affected is just 🤩

— Sally Lait (@sallylait) June 8, 2021

Check out @incident_io to manage incidents within slack. Awesome workflow and all the automation you ever wanted to bring more structure and fun into a potentially stressful situation. We'll start using it for every alert @giantswarm now.

— Timo Derstappen (@teemow) June 12, 2021

Super awesome demo of @incident_io from @sjwhitworth earlier. Really excited about what the team is doing to switch up incident management by bringing together tools we use & love into a streamlined workflow🔥🔥

— Nick Coates (@nickcoatesuk) June 10, 2021

What we shipped

• 💅 We've stopped Slack unfurling links when we post from @incident. We share all the context you need, so pulling it from an external link just cluttered the channel.
• 👷🏽‍♀️ We've been working on some big resiliency improvements; splitting parts of the app into distinct components to make us more reliable and tolerant of failure.
• 👷🏽‍♀️ We've been improving the state of our build and deploy pipelines. We can now test our code, and deploy it through to production in ~5 minutes - handy as we start ramping up on shipping ⚡
• 👷🏽‍♀️ We've added a number of alerts so we'll know quicker when things go wrong, and specifically what's up.
• 🐛 We fixed a bug with our statuspage integration which was causing errors if you opted to tweet the status update too. Thanks for the report Render!
• 🐛 We use idempotency keys to prevent duplicate items being added to the incident timeline, and one of our keys included the current time. Not a great choice, clearly, but so we're now using something reproducible.
• 🐛 We fixed a bug that showed the incident reporter as the person who generated the postmortem document. Now the correct person gets all the credit.

We now have a number of customers using us daily to respond to incidents, and we're seeing more of you leaning into improving your post-incident followup. This is an area where until now we've been heavily opinionated, without much room for customisation - something we always knew we'd need to change.

We want to make it easy to imprint your process directly into incident.io - we want you and your team to feel like this is your tool, not someone else's.

We've always been able to automatically generate incident documents (also often referred to as post-mortems) and timelines automatically, but now you can now customise the format to your heart's content.

What we shipped

• 🆕 Assigning someone a role in an incident automatically invites them to the channel. Thanks for the polish suggestion, Upvest!
• 🆕 We added lots of new logos to our website. Hello WorkOS, Gitpod, Livestorm and Prolific!
• 💅 We added typeahead support to all places where you can select a user in Slack. This means we can support much larger organisations, and make it easier to search.
• 💅 We shortened the message that we send when an incident is closed. It was a little long and in-your-face.
• 💅 When creating an action in the dashboard, we now sort the list of users that you can assign the action to by name to make it far easier to scan. Thanks for the prod, Truelayer!
• 💅 We flipped the ordering of ratings when giving feedback so it goes lowest to highest, instead of the reverse. It felt more natural, you know?
• 💅 We now only show you a nudge to customise your roles if you haven't already done so. You're clearly an expert, so we don't need to waste your time with the basics.
• 💅 The incident lead/commander is now always shown first when interacting with roles in modals or settings. Because they're important like that.
• 👷🏽‍♀️ We totally finished our migration to slash commands. Conversational UI, be no more!
• 👷🏽‍♀️ We added company information to our receipts and invoices. We're growing up. Thanks for the prod, Upvest!
• 🐛 If you tried to close an incident twice, in some circumstances, we'd post two messages. We now handle this idempotently.
• 🐛 We found a particular edge case where changing the incident severity via a modal after an incident was closed would remove the closed_at timestamp that we use for tracking. That was silly. It now doesn't do that. Thanks for helping us spot this one, Farewill!
• 🐛 We found a rare issue where we'd fetch the wrong message from Slack. This happens when we try and fetch a message that has been deleted. If it has, Slack will return the message immediately before the deleted message - which wasn't the message of interest. We've now fixed that! Sorry for the phantom bugs, Upvest and Bud!

In incident.io, roles are a first class citizen. We think of them as 'hats that you can wear' during an incident. We use them to make it clear:

• What the responsibilities are of a given role (e.g. the Communications Lead is responsible for internal and external communications)
• Who is currently playing that role (e.g. Pete)

Until now, we only supported a single role - the Incident Lead. We knew this was limiting, and our customers agreed, so we fixed it.

You can now add and customise your own roles to fit how you like to respond to incidents, attaching a description and instructions on how to play the role. Roles can be assigned and re-assigned during incidents by using /incident roles.

When someone is assigned to a role, we'll privately send instructions to the assignee to make it clear what they need to do, and help them get up to speed.

Head over to the Help Centre to learn how to use them, or join us in Slack to let us know what you think!

What we shipped

• 🆕 If you /zoom in an incident channel, we'll prompt you to add it as the call for the incident. Like magic.
• 🆕 If a user joins an incident channel, we'll now automatically acknowledge any open PagerDuty escalations they are assigned to. This’ll prevent those accidental escalations to others when you jump into incident mode, and saves you precious time and headspace.
• 🆕 We have a small-but-lovely Community where we share sneak previews of features, and get open feedback on the product. We now invite you to join it after you've closed your first real incident.
• 💅 If you make updates to the incident as you close it, we used to splurge a lot of additional updates into the channel, in addition to the closing message. We're now a lot quieter. Thanks for the prod, GoCardless!
• 👷🏽‍♀️ As part of our migration to slash commands, we removed all references to @incident in our codebase.
• 🐛 If you tried to close an incident when you'd already archived the channel, we would barf. Now we don't. Thanks, Loom!
• 🐛 In some circumstances, we weren't announcing when actions were completed. We now do.
• 🐛 Occasionally, we wouldn't update the incident announcement channel post after some updates via a slash command. We now ensure that we always update it, regardless of the command you use. Better to be safe than sorry.

When we started incident.io, we felt passionately that we should replicate the feel of human communication during an incident. Need to escalate to someone? Just talk to us with @incident escalate and we'll guide you through it. The drawback? We made a lot of noise in your incident channel 🙉

We've listened to your feedback, and today we're making the move to Slack's slash commands. Whilst it's just a single character, moving from '@' to '/' has a big impact. When you /incident escalate it makes no noise in a channel, and as an added bonus we can take you straight to the right place. It's a win-win!

As if that's not enough, we're also introducing the Incident Home. Can't remember the command for the status page? Just type /incident (or /inc if you're in a hurry) and we'll help you get to the right place.

What we shipped

• 🆕 When you pin images in an incident channel, we'll add them to the timeline. Worth a thousand words.
• 🆕 We added lots of new customer logos to our homepage. Hello, new friends! 👋🏼
• 💅 When you export actions to your issue tracker, we'll now show the link within the actions modal in Slack.
• 💅 We've added tooltips to the participants on the dashboard. Now you can see exactly what we mean by a 'Key Contact'.
• 💅 We've stopped sending incident action updates to the channel after the incident is closed. These updates aren't super helpful, and draw attention the channel when it's not needed.
• 🐛 If you split your severity definitions over multiple lines, we didn't display the text nicely in Slack. Now we do! Thanks Duffel!
• 🐛 We found an incident in the future on the Insights page. We can't predict future incidents (yet) — it was just a timezone issue.
• 🐛 If you swapped the names of your incident severities, we didn't always update them when you hit save. If it's not DNS, it's overly zealous database constraints.

Most of the energy around incidents is, quite rightly, focussed on coordination and solving the problem, but there's a tonne of value in what happens afterwards, too.

Whether it's to spot topics for a postmortem, or learn which parts of your incident response could be improved, just asking "how did we do" is a powerful tool.

Often we forget to ask, and it involves a lot of additional admin to follow up and collect responses, but from today, incident.io will handle it for you.

We'll automatically check in at the end of an incident and ask everyone how it went, with a few specific questions, as well as some more general prompts.

This is just a first pass, but we've already seen customers using it to gather ideas for improving their incident response. Let us know what you think!

Our main focus this week has been making sure it's easy to learn what incident.io is, and how it works. We've spent a lot of time working on great tooling for incident leads. However, by definition, there's only a single incident lead but many participants and observers. We want to make sure you have a great experience too!

As part of that, we've polished the messages that you get when joining an incident channel to make them clearer and more concise. Additionally, you can indicate whether you're there to help (we'll announce it in the channel if you are), there to observe, or if you want a quick introduction to incident.io.

We'll spend the rest of this week focusing on improvements to onboarding: look out for some new features coming out over the next few days!

What we shipped

• 🆕 Ever felt like your incident channel names didn't quite reflect who you were? I feel you. Well, incident channel name prefixes are now customisable: take your pick.
• 🆕 Want to write an update on your Statuspage without telling the world about it? You can now optionally set whether to deliver notifications to customers that have subscribed - previously, it was mandatory.
• 🆕 As part of our gradual but continual march to slash commands, we now support lead, rename, doc and call. Give it a try with /incident lead me. Only a few left to go now..
• 🆕 Until this point, we didn't have a Statuspage. As a business that aims to make it easier to respond when things go wrong, that was an omission. You can now find us at status.incident.io. We, of course, will be using our Statuspage integration to update it.
• 🆕 We added lots of new customer logos to our homepage. Hello, new friends! 👋🏼
• 💅 Statuspage components now have their component group included in their name, if they have one.
• 💅 If you added an action and assigned it to a user using /incident action, we'd send two separate updates into the channel. We've combined that into one update. Thanks for the feedback, Upvest!
• 💅 While we're on actions, we no longer post updates into the channel if you add new ones or update them after an incident is closed, keeping your channel history a little cleaner, once an incident is closed.
• 💅 The ID and name of the incident now appear in the browser title when viewing the incident homepage. Nice idea, Upvest and FACEIT! 🙏🏽
• 💅 When incident channels get archived, some features of the app aren't available. We used to tell you this, but not explain why. Now we do, avoiding a poor "computer says no" experience!
• 🐛 We used to return Statuspage component groups as a component that you could say was affected. That was a little strange, so we don't do that anymore. Thanks to Bud for noticing this one!
• 🐛 If you /incident summary Hello world, we used to discard Hello world. That now forms part of the update that appears in the modal. Sorry about that.
• 🐛 In the document export of the timeline, pinned messages would appear at the timestamp that you pinned thme, not the timestamp that the message occured at. That was silly, and it's now fixed. Thanks FACEIT!
• 🐛 If you had more than 100 PagerDuty services, we'd error when you tried to escalate. We now perform typeahead search, so it's easier for you to find the service of interest.

Our customers want a way to train people on incident.io, without unnecessarily worrying others in the organisation that things are going sideways or messing up their lovely incident library and statistics. Until now, you had to use normal incidents to do this. That's no good.

Well, enter test incidents! Test incidents are exactly the same as normal incidents, except:

• We won't announce them to the organisation in #incidents
• They're excluded from your incident library or statistics
• We disable some integrations like Statuspage, to avoid embarrassment

We think that they're best used for:

• Training your organisation to use incident.io
• Testing new features out
• Practicing your incident response on game days

Kick off an incident by /incident test and let us know what you think!

First impressions are important. We've seen more and more people using incident.io recently, which has been fun to see, so we've used this influx of new faces as a prompt to think about what we could do better here.

We're currently focussing on onboarding and adoption to make it easier for new users of incident.io to get up to speed, whether that's when you install us, or the first time you're pulled into an incident.

We'll have more to show next week, but we've already made a start, so you'll probably notice a few changes. If you spot them, let us know what you think!

What we shipped

• 🆕 When you install incident.io to your workspace, you now get a much clearer message that tells you what we've done, and what you should do next.
• 🆕 When you join the #incidents channel for the first time, we now introduce ourselves, and tell you what the channel is for. We do this privately, of course.
• 🆕 If you want to automatically have a call attached to your incident at kickoff, you can! Some folks have a consistent call always at the ready, and now we'll automatically add it to your incident for you (thanks to Bud for suggesting this!)
• 🆕 Fed up of typing /incident to kick off an incident? Too many characters and you're falling asleep by the end? Well, you can now type /inc to save you time, and effort.
• 🆕 Keen readers of The Changelog will have noticed we're starting to experiment with slash commands in incident channels. This week, we added support for a bunch of commands, including close, recap, statuspage and escalate. Try /incident recap for a quick spin!
• 💅 If you escalate using Opsgenie, we'll now stream updates into the channel for you.
• 💅 If you created an incident with /incident The website is down, we used to ignore The website is down and make you type it again. We now automatically populate that into the modal, to save energy for your precious little fingers. Thanks Ravelin, Bud and many others for this feedback!
• 👷🏽‍♀️ Our backend application is currently one large program, and we hope it stays that way. We're making some improvements to how we structure it to improve speed of development, reliability and correctness.
• 👷🏽‍♀️ We invested a bit of time in our build and deployment pipeline - getting it down from 10m to a far healthier 2m, every second counts!
• 🐛 If you logged in to app.incident.io on multiple Slack workspaces, you'd sometimes see an error when we thought you'd installed the app in a workspace, when you had not. We've now fixed that!
• 🐛 The biggest lie that software engineers tell themselves is that "we'll do pagination later". We uttered these words when building our PagerDuty integration to show you the list of services, and unfortunately, you were limited to 25. We implemented pagination, and now there are no limits!

In 2021, most companies have a status page set up to make it easier to publicly communicate the real-time status of their service to their customers.

However, when you're in the heat of an incident, it can be challenging to remember to update it. Where did we put the login, who has the right permissions, where do I go to update it? These are things we've been asked about time and time again by customers.

To help make things easier, we've brought one of the most popular status page providers out there - statuspage.io - right into your incident channels. Type /incident statuspage to notify customers of new incidents, provide updates to ongoing ones, and to let them know when the impact is over.

To get started using this for your incidents, head over to our shiny new documentation which takes you through how to install and use it.

Statuspage has a great feature inbuilt that allows you to pre-fill the impact, update text, and affected components. This makes it less stressful when figuring out what to write. We don't currently support this, but we hope to have it out in a couple of weeks. If this is something you'd like to see, let us know!.

What we shipped

• 🆕 We built a beautiful Help Centre. You can now learn all about what incident.io is, and how to use it.
• 🆕 We created the incident.io Community - a Slack workspace for us, and our lovely customers to hang out in. We'll use this to share early work, get feedback, and discuss interesting topics as a group.
• 💅 When you close an incident, we'll ask you to assign an owner to the postmortem. Every little nudge helps (thanks to Bud for this one!).
• 💅 You can hide a PagerDuty escalation policy or service from appearing in incident.io by adding the #incident-io-ignore tag anywhere in the description. This helps if you have test rotations, or a list of Very Important People that you'd never like to page (Bud again 🙌)
• 💅 We were a little short-termist when showing you your insights, defaulting to the last 7 days. We now show you 30! What will you do with your extra 23 days? (thanks for the feedback Upvest!)
• 👷🏽‍♀️ We're prototyping using slash commands to interact with the bot. This provides us with a lot more powers out of the box. We're seeing what it feels like first, before rolling it out more widely. Watch this space! 👀
• 👷🏽‍♀️ On the very rare occassion that we crashed, we'd return the default Heroku 'app crashed' page. We now return something more beautiful, and witty. We hope you never see it 🙈.
• 🐛 When adding an Opsgenie integration, we weren't as thorough as we liked with checking that you were on the right tier to have API access enabled (grumble, grumble). We now check that at integration time - sorry for the hassle, Ravelin!

We've been on a bit of a roll recently when it comes to adding integrations. Over the past month, we've added Opsgenie, JIRA, GitHub and Linear. Well now, Clubhouse gets to join the club!

You can use our Clubhouse integration to export actions from your incidents, to aid prioritisation, and make follow-up a breeze. When you've exported your actions, we'll continue to track and sync actions into incident.io, and link back to the incident from any issues we create, in case you need to reference it later.

One of the things we knew wouldn't last long was "open/closed" modelling of incident states.

That's not how any incident we've ever been part of has worked and although it got us off the ground, we've been embarassed about it ever since.

Your feedback told us the same thing, and we decided it was high time we fixed it.

Incidents can now be in several states:

• As soon as you kick off an incident, you'll be Investigating it
• Once you've figured out what's wrong, you'll be Fixing it.
• Once you think you've put a fix in place, you'll be Monitoring it.
• Once it's all over, the incident will be Closed.

These states can be controlled via Slack, and the timestamps can be updated via the dashboard after the incident if any adjustments are needed later.

We think they're really useful for clearly communicating the progress of an incident to your team and stakeholders. Additionally, capturing this data lays the foundation for lots of interesting and useful insights, and we look forward to sharing those with you in the near future.

What we shipped

• 🆕 You can add postmortem links explicitly now
• 🆕 You can create actions from the dashboard, so if you'd rather follow up there rather than in Slack, you can
• 💅 Some of our Slack notifications were incredibly unhelpful and just said "content cannot be displayed". We found a way to make them displayable. Thanks to the fine folks at Ravelin for spotting that one.
• 💅 We now use the issue reference in the "open in X" links for Jira, Linear, GitHub issues etc (thanks, Future!)
• 💅 Sometimes you just want to export a follow-up action without assigning it to anyone. Now you can.
• 💅 We now announce when actions are unassigned, with the option for someone else to pick them up.
• 👷🏽‍♀️ We're a little smarter at matching users in Jira to users in incident.io than we were last week - every little helps!
• 🐛 We've added action completion times and links to external issue trackers to the generated postmortem documents (thanks GoCardless!)
• 🐛 When you completed actions via the dashboard, the completed_at wasn't updating
• 🐛 For organisations with lots and lots of Jira users, we weren't returning them all when assigning followup actions. Now everyone can pitch in, which feels much fairer (thanks Future!)
• 🐛 We more accurately reflect the "completion time" for actions updated via Jira and the dashboard
• 🐛 If you talked to us in a thread, we responded in the main channel to let you know that doesn't work, which was weird. Now we keep the conversation threaded (thanks Ravelin!)

In the early days of building incident.io, we had a hunch that "storing a document" against an incident might be a useful thing, whether for post mortems, or for something else.

We've since had two bits of feedback repeatedly:

1. We only use to store the post mortem document
2. We'd like that to be more clearly reflected on our incident homepage

So, that's what we've done. The generic "document" link is no more, and in its place we now support a post mortem document explicitly. Great feedback, keep it coming!

Another mid-release update from us today. You can now export incident actions straight into GitHub Issues!

GitHub Issues provides a developer-friendly, lightweight way to track actions so it's no surprise it's been fairly high on the list of requests from customers.

Once exported, we'll continue to track and sync actions into incident.io, just like you can with our Jira and Linear integrations. We'll also link back to the incident from any issues we create, in case you need to reference it later. Happy GitHubbing!

We were a bit quieter last week because we've been heads down building and responding to feedback, so we're giving you a bumper update this week instead!

Hot on the heels of our PagerDuty integration, we got a lot of feedback from teams using Opsgenie, who felt a little left out.

We're delighted to say those days are over, and you can now escalate to your team via Opsgenie from right within your incident channels. Just like with PagerDuty, let us know who you need, and we'll handle the rest.

We've also got a few other nice integrations and improvements we'll be talking about more later this week.

What we shipped

• 🆕 Although it's only been out for a month, we beautified our escalation flow. It's easier to understand, and you can now page multiple people at the same time. Go forth and disrupt their sleep.
• 🆕 We added some of our early customers to our homepage. We feel very lucky to have you all 😻
• 💅 If you export an action to Jira, we'll attempt to match the Jira assignee to an incident.io user, so it's clear who's doing what. We'll also sync the time that the action was completed back to incident.io.
• 💅 If you're lucky enough to have your message pinned as part of an incident timeline, your lovely face will appear next to it.
• 💅 If you export your actions to Linear, GitHub Issues or Jira, assignees are now optional.
• 💅 Previously, the push notifications that you received from us would say "This content can't be displayed". That wasn't very good. We now try and put something helpful in every Slack notification that is sent on our behalf.
• 💅 We now announce when actions have been unassigned (thanks Farewill!)
• 💅 If you got a command wrong whilst talking to the bot in a thread, we'd embarrass you by responding in the main channel. We've recognised the error of our ways, and now respond to in a thread instead (thanks Ravelin!)
• 👷🏽‍♀️ We re-wrote a lot of how actions work under the hood to make them more resilient, and well-tested.
• 👷🏽‍♀️ We've started sketching how we'll build Custom Fields. We're excited for this one.
• 🐛 If you didn't have an incident in the past week, we'd assume that you'd never had an incident, and wouldn't show you any insights as a result. That was embarassing. That's now fixed.
• 🐛 We squashed a few small but pesky issues with our Jira integration, such as sporadic timeouts.
• 🐛 If you created an action that mentioned a Slack channel or user, it would look quite ugly in the dashboard. We've now cleaned that up, so it works how you'd expect.

A little mid-release update from us today. You can now export incident actions straight into Linear!

We're big fans, and selfishly we're pretty excited we can use this ourselves, as copying things over manually after incidents was getting a little tedious...

Once exported, we'll continue to track and sync actions into incident.io, and shortly we'll show you insights into how many actions you create, how many get followed up, and how quickly, too.

We'll also link back from tickets to the incident, in case you need to reference it later.

We want to meet you where you work, which means gluing together the tools you already use. We've got a long list of integrations planned, and Linear was getting a lot of demand, so we bumped it up the list.

Getting this kind of early access feedback to help us steer the product is really what drives us and makes building incident.io worth it, so thank you! If you have more integrations you'd like to see, drop us an email or let us know on twitter.

We bring you this update under the rays of some glorious spring sunshine 😎🌼.

One of the problems we hear time and time again is that following up on actions after an incident is harder than it should be. They get written down in slack or a post mortem document, maybe exported to an issue tracker, but more often than not, are then left to languish.

Without a clear link back to the incident that caused them, it's hard to track whether or not they get done, let alone see how long that takes and be able to report on it over time.

As a first step towards solving this, from today you can manage your actions via the incident homepage, and export them directly to Jira. We link them back to the incident that they were created from, and keep the state in sync for you.

Next up, we'll be building tools to help you make sure they get done, and surface insights so you can truly understand how quickly that's happening.

We've started with Jira, but Linear and others may follow soon. If you're keen to see that happen, let us know!

What we shipped

• 🆕 Money! You can now pay for incident.io through a shiny new billing page, and we'd very much appreciate it if you did.
• 🆕 Asking for help is now less noisy. If you mention /incident help, help will be returned in a modal that only you can see.
• 🆕 Actions are now exported to your postmortem document.
• 💅 When pinning a message to the timeline, we now include the name of the person that sent the original message, along with the person who pinned it. How lovely.
• 💅 We'll only send nudges when the channel appears to be active. This means we won't send repeated nudges in the channel, which makes us look a bit silly.
• 👷🏽‍♀️ We improved the healthchecks that happen before we deploy a change to production. Yay, resilience!
• 👷 We cleaned up a few unexpected frontend errors we were periodically seeing in production. They were small, and nobody noticed, but we knew, and we weren't having any of it.
• 🐛 If you escalated to the person that installed PagerDuty to incident.io, PagerDuty would automatically acknowledge the page, and not wake the person up. Unfortunately for the installer, they will now be woken up. Sorry, installer.
• 🐛 We got bitten by a classic "DivByZero" bug in our insights stats calculations. Easily missed, easily fixed 🙈
• 🐛 We weren't colouring your severities unless they matched the defaults, so if you used SEV1/SEV2/SEV3 they were all an uninspiring shade of grey in the dashboard. Now they're in technicolour, too!

"Resolving our incidents is only half the story"

That's what you keep telling us, and we couldn't agree more.

To really get the value from your incidents, you need to follow up effectively. Right now, we're focussed on building out the tools to help you do just that. We're starting with extracting the insights and learnings which, let's face it, you've already paid for. We want to help you get your money's worth.

A common solution we've seen (and used ourselves!) is what we at incident.io HQ refer to as "The Google Sheet of Doom". Often this is a manually compiled spreadsheet, containing some high level numbers on things like duration, severity and impact, maybe with some graphs and Pivot Tables thrown in at a stretch.

We think we can do it better, and do it for you. Today we're taking the first step in that direction by launching the first version of Insights, available for all incident.io early access customers.

We love our bot. It takes rote work off our hands, and let us focus on the things humans are great at. However, it's important that the incident channel remains clean and easy to scan, and it were getting a little loud for our tastes.

We started the week deep in thought, coming up with design principles for how the bot should interact. We'll be writing about these soon - stay tuned to our blog for more.

Armed with these principles, we built them into the product. The bot should be noticeably quieter. Let us know what you think!

What we shipped

• 💅 Incident status is presented in a modal, rather than a thread.
• 💅 If it's clear that an action has been taken, the bot will ack with emoji rather than repeating what's happened.
• 💅 When creating an incident, the message we show to the reporter is smaller, with some handy buttons right there in front of you. Why type, when you can click!
• 💅 When we detect a call link and offer the button to set it, we'll now clean up after ourselves and remove the original message
• 🐛 When adding a call link to the incident after we suggested it to you, the button wasn't going away. Now it does!

Do you ever find yourself rummaging through an incident channel for a dashboard someone shared, a runbook the team referred to earlier, or that documentation you were reading a few minutes ago?

We do — all the time.

From today, you can just type /incident links and we'll scurry off and find them all for you. Let us know what you think!

It's often the case that you reach the boundaries of your knowledge during an incident, and you need that special someone to help you out. We've all been there. However, getting help can be hit and miss: tagging people in Slack, or trying in vain to blow the dust off your PagerDuty login.

Fear no more - you can now page users and rotations directly in Slack via /incident escalate. We'll stream notifications and acknowledgements into the channel as they happen so you know who's on their way.

We've launched with PagerDuty to get ourselves comfortable with the flow, but we'll be launching Opsgenie and VictorOps soon!

What we shipped

• 🆕 Actions are now reflected in the timeline, so you can see exactly when something was picked up, put down, or passed along.
• 💅 If we can't render items in your timeline due to permissions issues, we'll let you know and if it's fixable, we'll tell you how.
• 💅 The link to the Slack channel from the incident homepage now refers to it by name, rather than being generically worded.
• 💅 If you link a document to your incident, we'll show that on the incident homepage, too.
• 👷🏽‍♀️ We have to fetch lots of data to render your incident homepage. We now do more of that fetching in parallel to save you time. What will you do with your extra milliseconds?
• 🐛 Severities no longer fail to update in the channel topic if you update incidents from an edit modal
• 🐛 We no longer sometimes shortchanging users in incidents by incorrectly labelling them as "observers", when they were picking up actions, resolving issues and generally being good eggs.
• 🐛 Renaming channels to really, really, really long names no longer fails to update the post in the incident announcements channel.

We've been getting tonnes of great feedback from our early access customers this week. We also said hello to 4 new ones. 👋🏼

Incidents can be fairly stressful, so the fact so many of you not only fixed your own bugs, but also helped us find and fix a few of ours is something we're incredibly grateful for.

We've been spending a bit of time thinking about how we can best share some of the longer term thinking we've been doing on what's coming next. This week we're really excited to launch our incident.io product roadmap, which lets you get a view into what we're up to.

There are upsides and downsides to sharing a product roadmap publicly, but we believe pretty strongly that:

1. Unless there's a strong reason not to, we'd prefer to default to transparency.
2. We're building this product with you, our customers, and we'll work better together if you can see what we're up to.
3. The feedback is clear, and we've got high conviction in what our customers want, so we're happy to write it down and talk about it openly.

We'll keep on shipping plenty of changes, improvements and fixes every week, but we'll use this as a way to keep you updated on some of the "big stuff" as it takes shape — enjoy!

What we shipped

• 🆕 You told us not having your emoji in the timeline was practically criminal. We wholeheartedly agree. They now render right there in the timeline.
• 🆕 You told us our Slack topics weren't, well, topical. We've revamped them so they're much snappier, and show much more useful information.
• 🆕 You asked for more nudges, so we've added a few for updating the incident summary and updating the status page which pop up after ~30 minutes or so.
• 🆕 We've added whereby as a recognised call provider.
• 🆕 When you ask for a status update, we now also give you the option to edit the incident directly from the message.
• 💅 Incident timelines now show a lot more useful information, which is great, but you told us they were starting to feel a little squished. They now have much more room to shine.
• 💅 Lots more incident information is now available tthrough the dashboard. Links to the Slack channel, any ongoing incident calls and more.
• 💅 Postmortem documents you generate automatically link back to the incident homepage.
• 👷🏽‍♀️ We launched a system to process events asynchronously (we use Google Pub/Sub, if you're interested). This means we can be more robust to failure, and automatically retry when things go south.
• 🐛 Our Content Security Policy headers were occasionally butting heads with the Avatar URLs Slack was returning. Your face should be smiling back at you in the nav bar once again.
• 🐛 Unpinned items no longer appear on the timeline.

This week was a particularly special week, as we onboarded our first 4 customers. 🎉

As such, we focused our time on helping them get onboarded, talking to them, and figuring out what we should build next.

What we shipped

• 🆕 You can now unassign actions from yourself. No need to be stuck with those pesky tasks - delegate!
• 🆕 We shipped a Security FAQ page, for all your security FAQ needs.
• 💅 If you create an incident from a message, we now ensure that the message you used is unfurled at the start of the incident for all to see.
• 💅 If you change your name in Slack, we reflect it in the product much faster. No identity crisis for you, my friend!
• 🐛 When using funky quote combinations in summary updates, we now strip them correctly for you.
• 🐛 Sometimes when you assigned an action to yourself, we'd update messages in Slack so it looked like you had also created the action. We fixed it so we don't re-write the history books.

We launched last week, and it went quite a bit better than we expected!

We've been inundated with support, feedback, potential customers and ideas for what to focus on next. However, pride comes before a fall, so we're being careful to keep focussed on talking to customers and building.

We spent the week putting together the final polish on the website before we went live. We've also been preparing the product ready to put it in the hands of early access customers, so we've been dotting the i's, crossing the t's and wrangling the experience to make it "just so".

What we shipped

• 🆕 You can now create incidents directly from messages and alerts.
• 🆕 Don't like our #incidents channel and want to name it #hamsters instead? You can now do so.
• 💅 Slack messages are now more nicely presented in the timeline, especially for links and user references.
• 💅 Durations appear more nicely formatted for incidents that stay open over 24 hours.
• 💅 Dashboard buttons don't resize weirdly on smaller screens any more - we hear you, Macbook Air crowd!
• 👷🏽‍♀️ We sped up our build and deployment pipeline. We can deploy code to production automatically in less than 5 minutes from merging a pull request. We hope Charity Majors would be proud.
• 👷🏽‍♀️ We made sure that we'd get paged and a Sentry error raised whenever something went wrong.
• 👷🏽‍♀️ We did a full security review of our application and cloud infrastructure.
• 👷🏽‍♀️ We made sure that we'd get paged and a Sentry error raised whenever something went wrong.
• 🐛 The incident bot no longer shows up as a participant on the incident homepage (we prefer to stay in the background).
• 🐛 The timestamp for the last update to the summary formats the date correctly.
• 🐛 Slack pins show up on the timeline in the order they were sent, rather than the order they were pinned.
• 🐛 The Settings dropdown used to stay open even when politely asked to close. We no longer ask politely.