Every few weeks somebody tells us they'll build this themselves. They usually can. The vendor APIs are good now, and a first version is about a week of work.
So here's how to build it. Then here's what you run into next, because that part nobody scopes.
How to build a v1
No paywall on this part. Claude is the worked example below; if you run several assistants, multiply the four pieces by the number of vendors and keep reading.
- Claude Code telemetry. In your Anthropic org settings, under Claude Code, push OpenTelemetry config to your users: an OTLP logs endpoint, a traces endpoint, your auth headers, and protocol
http/protobuf. That covers the CLI, the IDE and the desktop app. You need Owner permissions. - Cowork telemetry. Different screen, under Cowork monitoring, and a different protocol on purpose:
http/json, not protobuf. It also wants a recent Claude Desktop build. - Chat history. Organization settings, then Data and privacy, then Export data. Anthropic emails a link to a zip file. This one is an export rather than a feed, so "current" means whenever somebody last ran it. Enterprise plans can pull it through the Admin and Compliance APIs instead.
- Cost. A separate API again, with its own schema.
One practical note: if anything else in your stack already consumes OpenTelemetry, don't point two exporters at two destinations. Run a single collector and fan out from there.
What v1 gets you
A week of work buys real answers to the volume questions. Who has an account, who actually opens it, how often, on which surface, and what it costs.
A v1 that answers the volume questions earns its keep on its own. It settles the 200-seats-for-40-people question, finds the team that got licenses in March and never logged in, and puts a number on the bill before finance comes asking.
It also buys you the ability to spot-check individual runs, which is where it stops. As it turns out, most of these projects stall right there: you can see that people use AI, and you still can't say whether any of it works.
What v2 takes
The real work is getting from "usage went up" to "here's what to do Monday," and that's where the next eight items live.
- Classification. A transcript tells you a conversation happened. To learn anything you need every conversation labeled: which task, how complex, whether it worked, whether the person had to fight the model to get there. That takes a model pipeline with an eval set behind it, not a chart.
- Patterns instead of spot-checks. Nobody scans 40,000 conversations one at a time. You need clustering that surfaces the twelve things your team keeps doing, ranked by how much time they take, without a human sampling transcripts and guessing.
- People, skills and agents count as different units. A per-user chart quietly breaks the first time work moves to an agent, because the same task shows up as a service account at 3am. Deciding what counts as one person's output turns into a modeling problem.
- One story across surfaces. Chat, IDE, agents and Cowork describe the same person four different ways, in four schemas, on four release cadences. Stitching one identity across them consumes most of the ongoing maintenance.
- Something to compare against. Your dashboard will say usage rose 30%. It can't say whether 30% counts as good for a team like yours, because it has only ever seen you. Comparing your own teams against each other covers part of it, and the rest takes someone who has watched a lot of companies try this.
- A tie-out to outcomes. Time saved only proxies for value. The question that lands with a CFO joins usage to CRM data: did the accounts your reps worked with AI close faster, at better rates, and with fewer touches. That join is doable, and it takes longer than a week.
- A way to work with the data. Dashboards get opened during launch week and then live in a bookmark. Put the data behind MCP, so people can ask it questions inside the tools they already use, and you change who bothers to look.
- Change management. The last mile has nothing to do with analytics. Somebody has to take what the top quartile does, turn it into something everyone can run, and then get them to run it. A dashboard won't close that gap no matter what you build into it, because publishing and adoption are a different product.
Where that leaves you
Add it up honestly and it's classification models, an identity graph, a benchmark set, a delivery surface, and somebody who owns the outcome. Almost none of that is dashboard work, which is worth knowing before you scope it as a dashboard.
Some teams should build it anyway. If AI usage data sits close to what you sell, own it.
For everyone else the useful move is smaller. Build the v1, price the next eight items in engineer-months, then make the call with real numbers.
Don't start v2 until you've priced it. Most teams find they need three of the eight, not all of them, and a week of telemetry is what tells you which three.