Scoping week

We introduced On-call just a few months ago with a foundational set of features, marking only the beginning of our vision for the product.

Like many companies, we strive to plan our product roadmaps quarterly. On-call is no exception, but it presents many unknowns as a new product.

To address these uncertainties and maintain our momentum in development and delivery, we recently dedicated a week for the team to focus on scoping some of the key upcoming projects in our roadmap.

We split the team into pairs and assigned each pair to a particular project to reduce ambiguity, figuring out what we did and didn’t know, determining what was possible (and what wasn't), and ultimately seeing how we could tackle the project once it came around.

The primary goal was to turn broad themes of work into well-defined project plans that were technically scoped and sized. This approach aimed to help us build a roadmap that we could confidently commit to, ensuring the delivery of projects in a sensible order.

Why did we do it?

While we had already agreed on the themes of work for our roadmap, they were initially vague. For example, no one knew exactly what we might build for “insights”, making it impossible to estimate the effort required or understand the value it would deliver.

Anytime a project overruns, we look back to try and determine what impacted our timelines. In doing so, we found a few recurring factors:

Under-scoping work by starting too quickly without exploring technical implementation details.
Inaccurate estimates due to un-scoped work, resulting in the project being many times more expensive than anticipated.
Poor sequencing of projects; the team could have been more effective if they’d worked on things in a different order, allowing them to apply relevant learnings from an earlier project that had some dependencies with a later one.

With scoping week, we wanted to create space for the entire team to collaborate, building a roadmap that incorporated our product and customer expertise, aligned with our long-term goals, and was technically deliverable sustainably and effectively. Proper collaboration on planning requires dedicated focus, which is why we set aside a scoping week to engage the entire team in looking ahead, rather than just focusing on the immediate next step.

Our goals

The overarching goal was to transition from a vague idea like “we’d like to do something around insights” to a precise plan of “this is exactly how we want to do it”. This included defining both the product perspective and the technical elements of delivery.

We chose four themes to scope, which we determined would be most impactful to drive the usage and adoption of our on-call product.

Each team was then distributed across four sets of pairs, who were tasked with delivering the following:

A proposal document detailing what we want to deliver and how it would be delivered, covering both product scope and technical scope.
Engineering day estimates and ideal project size.
Clear next steps before the project kickoff.

Each theme could evolve into several projects, but each project had to meet these requirements and be ready for immediate pick-up after the scoping week.

Running the week

Preparation (Week before)

In advance of the week, our Product Manager Miia arranged a series of customer interviews and discovery sessions for each theme. Each theme had about ten interviews with a PM, designer, and rotating engineer. Miia prepared notes and analyses from these meetings to inform the team during scoping, avoiding delays from needing additional customer interactions.

Kick-off (Day 1)

We began with a full-team kick-off, structured as follows:

Recap team goals, mission, and KPIs.
Our designer, Sam, presenting concept designs for “On-call in 6 months”.
Recap roadmap themes and review discovery analysis for each theme.
Free-form "whiteboarding" and discussion on potential actions for each theme.

The pair assigned to each theme drove the discussion, aiming to capture the collective experience and ideas of the team. Each pair then started scoping and drafted a proposal to be reviewed by the team the next day.

Review (End of day 2)

After a day and a half of scoping, each pair presented their work to the group, focused on the product scope of the projects. The team provided robust challenges to ensure:

Everyone understood the specific project plans.
There was buy-in on the project's value.
Potential problems were identified early.
Familiarity with each pair's plans.
Any cross-over with other projects was flagged and resolved.

This early feedback was crucial to prevent investment in plans that wouldn’t work, ensuring alignment and clarity. Pairs then took this feedback and went back to fine-tune their documents.

Technical scoping (Day 3)

The third day was focused entirely on in-depth scoping of the specific projects decided on from the previous day’s review.

This meant diving deep into how things should work not just on the surface, but how it would be implemented in practice. At this stage, the pairs would dive into our past projects to find similar technical work or dive into the codebase to figure out which bit might be tricky to change.

Some of this technical scoping also fed back into the product scope, as pairs realized a particular thing might be difficult, and not worth the time.

Final call for estimations (Penultimate day)

As scoping drew to a close, each pair provided:

Total engineering days required for each project.
Ideal project size, considering team structure and parallelization.

This exercise ensured detailed planning, with estimations reflecting the full scope of work.

Roadmap sequencing (Final day)

With a list of projects and estimations, we could confidently assemble a roadmap. Our methodology included:

Arranging projects by ideal delivery milestones
Ensuring sequencing made sense for the team, considering expertise and complexity
Planning for capacity and bandwidth, with constraints to avoid overloading and ensuring buffer periods

The final step involved presenting the roadmap to the team for final adjustments based on feedback.

The outcome

The scoping week was a resounding success. The resulting roadmap was impactful, and the team felt more invested in the upcoming quarter’s work. We had specific plans and designs, replacing vague ideas and ensuring everyone was on the same page.

The collaborative effort between product, design, and engineering functions led to better outcomes, with shared perspectives enhancing the planning process. This holistic approach also meant better project integration, avoiding redundant efforts and ensuring cohesive project delivery.

For a team that is still very recently out of "ship a new product" mode, scoping week also provided a good reset; it allowed the team to step back and plan projects in the same way we plan projects in the rest of our product. Towards the end of building On-call, a lot of work went into reliability and product polish—the kind of work that has a different shape to normal product work—which meant the team had lost some of that project muscle. With dedicated space for scoping, we all got reminded of what a good project plan looks like.

Overall, scoping week has set a new standard for our project planning, and we look forward to the positive impact it will have on our future work. We expect that our detailed preparation and collaboration will result in smooth project execution and meaningful outcomes. In three months, we will reflect on the success we aimed for and see how well we executed our plans. 🔥