Copilot,
Traced.
Your AI pair programmer is a distributed system. It has been emitting OpenTelemetry this whole time.
Three questions, and I don’t
think anyone here can
answer them.
How many tokens?
Not roughly. The actual number your last turn sent to the model.
What did it cost?
In credits. For that one turn. On that one repo.
Was any of it cached?
Or did you pay full price to re-send a prompt the model had already seen?
Your checkout service is better
instrumented than the thing
writing your code.
Fully observable
Traces. RED metrics. Structured logs. An on-call rotation. A dashboard someone actually looks at. Alerts that page a human at 3 a.m.
“Feels a bit slow today”
No traces, no baseline, no record of which model ran or what it spent. When it degrades you find out on Slack, from a colleague, in a shrug.
There’s a distributed system
running on your laptop, and
you’ve been calling it autocomplete.
A planner that calls a model. A model that calls tools. Tools that call your file system, your shell, an MCP server two networks away. Retries, timeouts, a cache, and a bill. You have instrumented this exact shape a hundred times.
This is the whole thing.
No SDK to install. No agent to inject. No proxy. Copilot speaks OTLP natively — you just tell it where to send it.
{
"github.copilot.chat.otel.enabled": true,
"github.copilot.chat.otel.exporterType": "otlp-http",
"github.copilot.chat.otel.otlpEndpoint": "http://localhost:4318",
"github.copilot.chat.otel.captureContent": false
}
setx COPILOT_OTEL_ENABLED "true" setx OTEL_EXPORTER_OTLP_ENDPOINT "http://localhost:4318" # Leave OTEL_SERVICE_NAME unset. That single omission # is what keeps VS Code and the CLI apart on the board.
Which surfaces actually emit? Fewer than you’d hope.
One turn is a trace you already know how to read.
Click a span. live
Why one moved byte costs you 21,000 tokens.
Cache hit rate is a leading indicator.
It is the closest thing Copilot has to a health metric — and it moves before anyone complains.
Your prompt is stable
A high hit rate is a proxy for a well-behaved system prompt, a stable tool inventory, and sessions that aren’t thrashing their context window.
Cost, before the invoice
Cached input bills at a fraction of fresh input. Hit rate falls today; the finance conversation happens next month.
Latency, before the complaints
A hit shaves most of the time-to-first-token. "Copilot feels sluggish" is usually a hit-rate chart nobody was looking at.
One wire, and four reasonable places to point it.
Build your pipeline. Take the config. live
service.name is not “where I was sitting”.
This is what falls out.
All / VS Code / Copilot CLI. Every panel below re-queries.
The same questions, per surface.
Four ways to destroy your hit rate. live
Now multiply it by your team. live
Cost is on the span now — you don’t have to infer it.
| attribute | what it gives you |
|---|---|
| github.copilot.nano_aiu | AI units × 10⁻⁹ — divide by 1e9 for the credits figure the CLI prints |
| github.copilot.cost | Request cost as reported by the service |
| github.copilot.server_duration | Server-side ms, versus the span’s wall clock |
| gen_ai.response.time_to_first_chunk | TTFT in seconds, streaming |
| github.copilot.turn_count | LLM round trips in the session |
Grafana unit si:nAIU — an SI nano prefix on a custom AIU unit — rescales 21758500000 to 21.8 AIU with no transformation at all.
Content capture off ≠ anonymous.
The rollout path stopped being a hack.
Monday morning, in order.
Twenty minutes: see your own data
docker compose up -d, paste four settings, use Copilot normally for an hour, open localhost:3001. Nothing leaves your laptop. No cloud account, no approval, no meeting.
One afternoon: find your worst prefix
Sort by cache_creation.input_tokens. Look for a repeating high value — that’s your volatile token. Overlay the session.* events to see whether it’s compaction or something you wrote.
Then, and only then, talk about a fleet
Put a collector in the path, drop enduser.pseudo.id, and roll it out with managed settings — with a baseline you can point at, and an answer ready for “are you watching me?”
Six traps, pre-sprung.
| trap | what happens · what to do |
|---|---|
| Traces-only pipeline | Copilot emits metrics and logs too. No metrics: pipeline in your collector and they’re silently dropped. Tempo can’t store metrics at all. |
| Protocol mismatch | The CLI defaults to http/json, VS Code to http/protobuf. Grafana Cloud’s gateway wants protobuf — set it explicitly. |
| OTEL_SERVICE_NAME | Set it and both surfaces collapse into one name. Leave it alone. |
| Managed headers | Delivered to the Chat extension exporter only, never the agent host — deliberately, to keep tokens out of subprocesses. An in-network collector side-steps it. |
| Workspace settings | Use User settings. A committed workspace setting points your teammates’ telemetry at your endpoint. |
| Reload | The agent host reads config at start-up. Changed a setting and see nothing? Reload the window before you debug anything else. |
We let the tool that writes
our code be the one thing
we never measure.
It has been telling us the whole time — tokens, cost, latency, every cache miss and the reason for it. Four settings and a docker compose. Then it stops being a feeling and starts being a number.
Take it with you.
github.com/webmaxru/
copilot-opentelemetry
Docker Compose for the local stack, both collector configs, Azure provisioning scripts, and the two dashboards — TraceQL for Tempo, KQL for Azure Monitor. MIT licensed.
| OTel GenAI semantic conventions | opentelemetry.io |
| Monitor agent usage with OpenTelemetry | VS Code docs |
| Copilot CLI — OpenTelemetry monitoring | docs.github.com |
| Enterprise-managed OTel export | github.blog changelog |
Questions — and I’d genuinely like to hear what your hit rate turns out to be.