GitHub Copilot

Trace GitHub Copilot sessions in VS Code and the Copilot CLI in Phoenix for full observability.

Trace your GitHub Copilot sessions — in VS Code or the CLI — in Phoenix with the coding-harness-tracing toolkit. Every turn shows up as a trace, with each tool call and any subagents captured as nested spans and grouped into its session. No application code changes required: the toolkit hooks into Copilot’s own events and streams OpenInference spans to Phoenix. VS Code uses per-event hook files; the CLI uses a single hooks.json.

Launch Phoenix#

The fastest way to get started with Phoenix is by signing up for a free Phoenix Cloud account. If you prefer, you can also run Phoenix in a notebook, self-host it, or use it directly from your terminal.

Go to the settings page in your Phoenix instance to find your endpoint and API key. A self-hosted Phoenix defaults to http://localhost:6006; the API key is only required when auth is enabled.

Install#

Copilot hooks are installed at the project level under .github/hooks/. Run the installer from the root of each repository where you want tracing.

The curl installer is the simplest — it runs a short wizard that saves your Phoenix credentials for you. Use a local clone if you’d rather run the installer from a checkout of the source.

Curl installer (recommended)#

macOS / Linux:

cd /path/to/your/projectcurl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- copilot

Windows (PowerShell):

cd C:\path\to\your\projectiwr -useb https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.bat -OutFile $env:TEMP\install.bat& $env:TEMP\install.bat copilot

Local clone#

git clone https://github.com/Arize-ai/coding-harness-tracing.gitcd /path/to/your/project/path/to/coding-harness-tracing/install.sh copilot         # macOS / Linux\path\to\coding-harness-tracing\install.bat copilot        # Windows

The installer prompts for your backend — select Phoenix, then enter your endpoint and optional API key — and your project name, writes them to ~/.arize/harness/config.json, and registers VS Code (.github/hooks/*.json) and Copilot CLI (.github/hooks/hooks.json) hooks in the current project.

Configuration#

Credentials live in ~/.arize/harness/config.json and apply across all projects. Per-project environment variables override config.json and can be set in your shell profile.

export PHOENIX_ENDPOINT="http://localhost:6006"export PHOENIX_API_KEY="<your-api-key>"   # optional, only if auth is enabledexport PHOENIX_PROJECT="copilot"export ARIZE_TRACE_ENABLED="true"

On the Phoenix backend, set the project name with PHOENIX_PROJECT (or PHOENIX_PROJECT_NAME); ARIZE_PROJECT_NAME is Arize-only and ignored here. ARIZE_TRACE_ENABLED is a backend-agnostic harness setting and keeps the ARIZE_ prefix regardless of destination.

Redaction controls#

Each ARIZE_LOG_* flag accepts "true" or "false" and defaults to "true". Set to "false" to opt out per category:

export ARIZE_LOG_PROMPTS="false"export ARIZE_LOG_TOOL_DETAILS="false"export ARIZE_LOG_TOOL_CONTENT="false"

Observe#

Once tracing is enabled, Copilot activity from both VS Code and the CLI streams to Phoenix. You’ll see:

• Turn traces — each prompt → response cycle as a parent span
• Tool spans — nested spans for each tool call with inputs, outputs, and duration
• Subagent spans — activity from any subagents Copilot spawns (VS Code)
• Error spans — errorOccurred events from the CLI
• Session grouping — all turns from the same session grouped by session_id

Reference#

For the full list of environment variables, default file paths, and troubleshooting steps, see the Copilot tracing README.

Uninstall#

Run from the project root where you installed the hooks:

curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- uninstall copilot

Resources#