Dashboards aren’t (quite) dead

Historically, non-technical stakeholders would’ve had most of their data questions answered either through pre-built dashboards or by asking their Data team (or equivalent).

Self-serve analytics tools went a step further by offering safe, governed datasets built by Data teams which let non-technical users dig into data without having to worry about how it joins together, how metrics like “revenue” are defined, and so on.

But now, anyone with an LLM (with data access) can not only run analyses via a few simple prompts - they can completely customise what the output looks like.

As a result, Data teams have had to implement reliable interfaces between LLMs and data by exposing self-serve tools via native LLM connections, building their own set of analytics instructions for LLMs, or a mix of the two.

All of this leads to a fairly obvious question: if any business user can (reliably) answer any question they want, on demand, with bespoke outputs, and even combine it with qualitative information like deal notes / Slack messages / Notion docs etc., why bother having dashboards at all?

It comes down to consistent views of your data, and, increasingly, to dashboards being useful inputs to LLM workflows, not just read-only outputs.

LLMs can give you inconsistent (but correct) results

There’s a lot of material out there on how to get an LLM to answer data questions correctly, but it mainly boils down to business context.

Point an LLM at your data warehouse, without any guidance, and ask it “what was our deal win rate last month?” and you’ll likely watch it mull over things like deal stages, what types of deal to include, and make a best guess at what it thinks is the most “correct” answer.

This isn’t a new problem, nor an unsolved one: table & column descriptions combined with a well-constructed semantic layer (table joins, primary keys, metric definitions) go a long way towards both humans & LLMs answering data questions consistently & correctly.

But this doesn’t change the fact that even with the same guidance, pulling the same metrics (correctly), that you can end up with different stories from the same data. A semantic layer makes sure everyone calculates win rate the same way, but it doesn’t decide which cut of win rate the business actually looks at each month.

Let’s extend the deal win rate example:

CEO: pulls win rates, monthly, across all types of deals.
Sales leader: pulls win rates, weekly, split by new vs. existing business.
Marketing leader: pulls win rates, weekly, for marketing-sourced new business.

All 3 approaches are valid, and all calculate win rates the same way. Yet, if you asked each of them “what have our win rates been over the past month?” you will likely end up with 3 different answers.

LLMs can remove the friction to get correct answers from your data, but you risk shifting this friction downstream to (in this case) 3 leaders in a room trying to reconcile why their win-rate numbers don't line up.

Dashboards help provide consistency that LLMs can lack

A common pattern to solve for consistency is to provide LLMs with example queries: if a user asks X then you probably want to run query Y.

Dashboards are just an extension of this: a “deal review” dashboard that contains a visual of win rates over time cut by N-dimensions is essentially just a saved SQL query, run & reviewed periodically, that the CEO / Sales / Marketing have agreed is useful to run the business with.

That agreement is the part an LLM can't (and shouldn’t) reproduce. It can compute any metric cut you ask for, instantly, but it can't decide which cut the business runs on: the dashboard is where that decision lives.

From a technical standpoint, you can also think of dashboards as the top level of curation of data.

More “curated” here means a stronger signal (both to a human and an LLM) on how to analyse data if you were asked a question with no prior business context (e.g. deal win rates):

Dashboards: How do we currently analyse this type of data? Which is the main one leadership use to look at this on a regular basis?
Previous queries & analysis: Have we previously analysed this type of data?
Semantic logic: If I’m running an analysis from scratch, how should I analyse this type of data?
Transformation logic & below (SQL, column descriptions, rows of data etc.): If I haven’t got any information on how to analyse this type of data, what do I think would be the most sensible way of doing so?

Dashboards are still useful as a consistent (and visually represented) set of curated queries, that both humans and LLMs can use as the starting point for “here’s how we actually analyse a particular set of metrics”.

Watch out for clutter

The blocker, historically, with dashboards was the time taken for a Data team to build it (especially when compared to the speed of LLM-based analysis). Modern BI tools are drastically reducing the effort required to build & maintain dashboards, which is a double-edged sword.

If it's trivial to spin up ten near-identical "win rate" dashboards, and neither a human nor an LLM can tell which is the source of truth, then the consistency problem just resurfaces in a different form.

Curation, again, is key. Make it clear which dashboards are the authoritative ones (label with “Data team approved”, organise key dashboards into top level folders).

It’s worth stating that keeping dashboards well curated is a really hard problem to solve. There are often cases that sit between a one-off analysis and something like a polished key metrics dashboard, such as tracking a set of success metrics to monitor a new product launch.

Not everything needs to be a dashboard. A useful proxy is to ask yourself “If this was broken in ~1-2 months’ time, would it be worth fixing?”. If not, it’s worth having a route to avoid storing these types of analysis alongside more curated, higher polish dashboards.

This could be subfolders in your BI tool for more ad-hoc work, or even lightweight alternatives to dashboards that offer visuals but are lower friction to construct.

Dashboards can be powerful LLM inputs

Whilst dashboards offer consistency, they aren’t without drawbacks:

It’s increasingly common for both technical & non-technical users to run the bulk of their day-to-day out of LLMs, and don’t want to switch between interfaces.
They’re often inflexible for end users, and it’s generally not easy to dive into the why behind a trend in the data.
They often lack qualitative data sources that makes analysis with LLMs so powerful.

The last point is particularly important, and we can illustrate this with a real world example!

Below is a schedule health factor metric for a customer over time that exists in our customer health dashboard. It’s a calculated metric between 0-100% based on how well we think a customer is using the scheduling part of our On-call product, which feeds into our overall customer health score.

This isn’t the full picture - we’ve only included one variable out of several that go into this metric - but imagine as a Customer Success Manager that you’d want to know why the customer is now using rotations more extensively in our product:

You could always speak with the customer directly - but this doesn’t scale well for lots of small questions.
You could also build tables / visuals into a dashboard for every possible follow-up question - in this example a table of the individual schedules - but this can quickly become cluttered, and not all information renders nicely in a dashboard form (e.g. JSON).
You can also drill into the rows of data behind each datapoint: but this is still a manual process that you need to repeat for every follow-up question you want to ask.

If we want to dive deeper into a trend on a dashboard, why not treat the dashboard itself as a source of information for an LLM in the same way you would Slack / Notion etc.? This unlocks more powerful workflows:

Using LLMs to combine qualitative & quantitative data isn’t new - but it’s a great way to leverage the consistency that a dashboard provides combined with the flexibility to dig 2-3 layers deeper on demand, and only dive into the areas worth exploring further.

Closing thoughts

Dashboards aren’t (quite) dead - but they’ve changed in purpose. They matter less as the sole place people go to look at charts, and more as the curated, trusted layer that keeps both humans and LLMs telling the same story from the same data.

There will always be a need for a consistent set of metrics, shown a certain way, for decision-making stakeholders to use on a regular basis. Dashboards still fit this purpose really well.
Dashboards can be thought of as the top level of curation for an LLM, and a useful starting point - essentially a set of queries that demonstrate how you actually analyse a set of metrics.
The interface people use to consume dashboards is less important. It should be flexible, and you should meet your users where they work.
LLMs have made dashboards far cheaper to build and maintain, but they only deliver consistency, or work well as LLM inputs, if it stays obvious which ones are authoritative.

Build fewer, name them well, and treat the best ones as first-class context for your LLMs.