OpenTelemetry · GitHub Copilot

You can't tune
what you can't see.

Turn GitHub Copilot's OpenTelemetry into one Grafana dashboard — prompt‑cache hit rate, tokens, latency and tools — across VS Code and the CLI. Runs on your laptop; scales to a fleet.

View on GitHub →
prompt cache · cache_read vs cache_creation 63%hit rate
cache_read (hit) cache_creation (miss)
VS Codeservice.name=copilot-chat Copilot CLIservice.name=github-copilot
The idea

Two surfaces, one vocabulary — so one dashboard.

VS Code Copilot Chat and the Copilot CLI both emit the OpenTelemetry GenAI conventions. The attributes are identical; the surfaces differ in service.name. Turn that into a dropdown and two data streams become one board.

VS Code Copilot Chat
resource.service.name = copilot-chat

Enable with four github.copilot.chat.otel.* user settings, or the same env vars.

GitHub Copilot CLI
resource.service.name = github-copilot

Enable with OTEL_* env vars. Same telemetry engine, shared with the Copilot SDK.

Copilot surface selector → All (VS Code + CLI) · VS Code · Copilot CLI. Keep OTEL_SERVICE_NAME unset so each surface keeps its own name.
Two axes, not one (updated August 2026) → surface is resource.service.name (copilot-chat · github-copilot · claude-code), agent is gen_ai.agent.name / gen_ai.agent.id. A CLI session launched inside VS Code emits under both names — so the dashboards now ship a Turns by Agent panel to tell them apart.
The honest map

Which Copilot surfaces emit OpenTelemetry?

Not all of them — yet. This project is built and verified on the two you can collect from today — VS Code and the CLI (the desktop Copilot app rides on the CLI runtime) — plus the SDK for apps you build. JetBrains joined the list in July 2026. No overclaiming.

✓VS Code Copilot Chatsettings, OTEL_* env, or managed settings
✓GitHub Copilot CLIOTEL_* env vars
✓Copilot app (desktop)own policy, CLI runtime → github-copilot
✓Copilot SDK — for apps you buildTelemetryConfig
✓JetBrains plugins · new July 2026Settings → Tools → Copilot → Chat · service.name not yet verified
✓Claude agent / Claude Code via Copilotcopilot-chat, claude-code
—Visual Studio extensionnot today
—Cloud coding agent (opens PRs)server-side; client counters only
JetBrains caveat. The July 2026 plugin release added an OpenTelemetry export panel, but GitHub hasn't documented which service.name it reports and we couldn't verify it firsthand — so the dashboards don't ship a JetBrains option yet. Point it at the same collector, run { } | count_over_time() by (resource.service.name), and add whatever name shows up.
What you can monitor

Traces, metrics, and events — the questions they answer.

Every interaction is a span tree: invoke_agent → chat → execute_tool. All of it follows the GenAI conventions, so the same queries work on both surfaces.

cache

Prompt‑cache efficiency

Hit vs miss over time, per model, per surface.

gen_ai.usage.cache_read
gen_ai.usage.cache_creation
cost

Tokens & cost

Input/output/cached tokens, plus per-call cost and AI credits.

gen_ai.usage.*_tokens
github.copilot.aiu
legacy: nano_aiu ÷ 1e9
latency

Speed

Duration and time‑to‑first‑token, sliced by model.

gen_ai.client.operation.duration
time_to_first_chunk
tools

Tool usage & reliability

Which tools run, how often, how slow, how often they fail.

execute_tool · gen_ai.tool.name
outcomes

Adoption & impact

Edits accepted, lines shipped, edit survival, thumbs, PRs.

edit.acceptance · lines_of_code
edit.survival · pull_request
reliability

Errors & sessions

Error types, stuck/aborted sessions, context compaction.

error.type · session.abort
why

Why the cache missed

Truncation and compaction rewrite the prompt prefix — the direct cause of a miss. Span events tell you when.

session.truncation
session.compaction_start / _complete
See it

One board. Flip the selector.

Cache calls and token volume, models, tools, and raw traces — filtered to VS Code, the CLI, or both. The August 2026 compatibility release adds the actual prompt-cache token hit rate, current/legacy AI-credit handling, corrected agent grouping, and exported truncation/compaction events.

localhost:3001 · GitHub Copilot OTel — VS Code + CLI
Grafana dashboard showing both Copilot surfaces: total LLM calls, cache read vs creation tokens, calls by surface (copilot-chat and github-copilot), calls by model, top tools, and a trace table.

The All (VS Code + CLI) view, running on local Grafana + Tempo.

Where the telemetry lands

From a laptop to a fleet — four backends.

Same surfaces, same dashboard. Start local and offline; grow into your Azure tenant or Grafana Cloud. None of them needs a paid Grafana instance.

A

Local

Docker: Grafana Tempo + Grafana. Offline, private.

Free · local
B

Azure · local collector

Collector fans out to Tempo and Application Insights.

Azure metered
C

Azure Container Apps

Cloud collector, scale‑to‑zero. Scrubs before App Insights storage.

Scale-to-zero
D

Grafana Cloud

Point straight at managed Tempo. Nothing to run.

Free tier
Rolling it out to a team (updated August 2026) → GitHub now ships a supported enterprise managed settings path with a dedicated telemetry block — enabled, endpoint, protocol, lockCaptureContent, resourceAttributes and headers (that's where the collector token goes). Delivered via native MDM, a server-managed managed-settings.json, or a file on disk — no more pushing raw env vars. One warning: don't set telemetry.serviceName fleet-wide, or every surface collapses into one name and the surface selector stops working. The repo has ready-to-paste samples per backend. The current support matrix documents the telemetry key for VS Code, Copilot CLI, and JetBrains, not the desktop app or cloud coding agent.
Your prompts stay yours

Private by default.

The question that decides whether you can roll this out — answered plainly.

→

Telemetry goes to the endpoint you configure. OTLP export does not create a second GitHub-controlled copy. The operator of your chosen backend still receives it — your team for local/Azure, or Grafana Labs for Grafana Cloud.

→

No prompts, code, or responses by default. Content capture is off — the GenAI conventions' deliberate opt‑in. You get metadata: model, token counts, durations, tool names, cache hits/misses.

→

"Content off" isn't "anonymous." Prompts, responses, system instructions, tool schemas/arguments/results are opt-in. Metadata still includes a CLI pseudonymous id when available, agent identity, invoked skill names/paths, tool metadata, and VS Code repo/branch/commit/org.

→

The collector is your minimization control point. Option B drops the pseudonymous id locally before cloud export. Option C strips span, content, repository, and skill-event attributes at the Azure collector before Application Insights stores them; the raw OTLP request has already left the device.

→

You choose where data lives. On your laptop, in your Azure region (pick an EEA region for residency), or Grafana Cloud's SaaS. Content capture, if you ever turn it on, only against a backend you own.

Start in five minutes

Clone, compose, chat.

The local path (Option A) needs no cloud account and costs nothing.

1

Bring up the stack

Grafana Tempo receives OTLP; Grafana serves the dashboard on port 3001.

# clone, then
docker compose up -d
2

Point Copilot at it

Four keys in your VS Code User settings — and the CLI reads the same via env vars.

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "otlp-http",
  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4318",
  "github.copilot.chat.otel.captureContent": false
}
3

Use Copilot, then watch

Ask Copilot Chat a few questions or run copilot, then open the board and flip the surface selector.

# open
http://localhost:3001  # → GitHub Copilot OTel — VS Code + CLI
Get it on GitHub →