OpenInference OTEL Tracing#

This documentation provides a guide on using OpenInference OTEL tracing decorators and methods for instrumenting functions, chains, agents, and tools using OpenTelemetry.

These tools can be combined with, or used in place of, OpenTelemetry instrumentation code. They are designed to simplify the instrumentation process.

Let your coding agent set this up. Works with Claude Code, Codex, Cursor, and OpenCode.

Start Phoenix

In one terminal, start Phoenix and leave it running. It serves at http://localhost:6006:

uvx arize-phoenix serve

Already have a Phoenix deployment? Skip this step. px setup can point at any running Phoenix, including a self-hosted one.

Add tracing with a coding agent

In a new terminal, from your app’s root directory, run one of these:

npx -y @arizeai/phoenix-cli setup

npm i -g @arizeai/phoenix-cli && px setup

px setup hands the instrumentation to your coding agent and waits until a real trace arrives. It can also install the Phoenix skills so your agent can query what you capture. See Connect a project with px setup for the full flow, CI usage, and unsupported agents.

Installation#

Ensure you have Phoenix OTEL installed:

pip install "arize-phoenix-otel>=0.16.0"

Note

Version arize-phoenix-otel>=0.16.0 is required for the examples on this page. Starting in 0.16.0, phoenix.otel re-exports the OpenInference context managers (using_session, using_user, using_metadata, using_tags, using_attributes, using_prompt_template, suppress_tracing) and semantic conventions (SpanAttributes, OpenInferenceSpanKindValues, OpenInferenceMimeTypeValues) directly — no need to also install openinference-instrumentation or openinference-semantic-conventions.

On older versions, import the same symbols from openinference.instrumentation and openinference.semconv.trace instead.

npm install @arizeai/phoenix-otel

@arizeai/phoenix-otel re-exports the OpenInference tracing helpers and context utilities, so Phoenix users can register tracing and add decorators or span helpers from the same package.

For detailed API documentation, consult the respective documentation sites.

Phoenix TypeScript API

OpenInference JavaScript API

Phoenix does not ship a phoenix-otel-go wrapper. Install the OpenInference Go semconv + instrumentation packages plus the stock OpenTelemetry Go SDK:

go get github.com/Arize-ai/openinference/go/openinference-semantic-conventionsgo get github.com/Arize-ai/openinference/go/openinference-instrumentationgo get go.opentelemetry.io/otelgo get go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttpgo get go.opentelemetry.io/otel/sdk

The semconv package provides typed constants for the wire-format-stable attribute keys; instrumentation provides context-attribute helpers (session, user, metadata, tags, suppression).

Setting Up Tracing#

from phoenix.otel import registertracer_provider = register(protocol="http/protobuf", project_name="your project name")tracer = tracer_provider.get_tracer(__name__)

import { register } from "@arizeai/phoenix-otel";const tracerProvider = register({  projectName: "my-app",  url: "https://your-phoenix.com",  apiKey: process.env.PHOENIX_API_KEY,});

Configure the OpenTelemetry Go SDK with an OTLP/HTTP exporter pointed at Phoenix, then obtain a tracer. For a remote deployment, set PHOENIX_COLLECTOR_ENDPOINT and, if it has authentication enabled, PHOENIX_API_KEY.

import (    "context"    "os"    "strings"    "go.opentelemetry.io/otel"    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"    "go.opentelemetry.io/otel/sdk/resource"    sdktrace "go.opentelemetry.io/otel/sdk/trace"    semconvotel "go.opentelemetry.io/otel/semconv/v1.26.0")func phoenixTraceEndpoint(endpoint string) string {    endpoint = strings.TrimRight(endpoint, "/")    if strings.HasSuffix(endpoint, "/v1/traces") {        return endpoint    }    return endpoint + "/v1/traces"}ctx := context.Background()opts := []otlptracehttp.Option{    otlptracehttp.WithEndpoint("localhost:6006"),    otlptracehttp.WithURLPath("/v1/traces"),    otlptracehttp.WithInsecure(),}if endpoint := os.Getenv("PHOENIX_COLLECTOR_ENDPOINT"); endpoint != "" {    opts = []otlptracehttp.Option{        otlptracehttp.WithEndpointURL(phoenixTraceEndpoint(endpoint)),        otlptracehttp.WithHeaders(map[string]string{            "Authorization": "Bearer " + os.Getenv("PHOENIX_API_KEY"),        }),    }}exporter, _ := otlptracehttp.New(ctx, opts...)res, _ := resource.New(ctx, resource.WithAttributes(    semconvotel.ServiceName("your project name"),))tp := sdktrace.NewTracerProvider(    sdktrace.WithBatcher(exporter),    sdktrace.WithResource(res),)otel.SetTracerProvider(tp)tracer := otel.Tracer("your project name")

Using Helpers#

Your tracer object can now be used in two primary ways:

1. Tracing a function#

@tracer.chaindef my_func(input: str) -> str:    return "output"

import { traceChain } from "@arizeai/phoenix-otel";const myFunc = (input: string): string => {  return "output";};const tracedFunc = traceChain(myFunc, { name: "my-func" });tracedFunc("input");

Go has no decorator syntax. Wrap the function body in a tracer.Start span and set input / output / span-kind attributes manually using the OpenInference Go semconv constants:

import (    "context"    "go.opentelemetry.io/otel"    "go.opentelemetry.io/otel/attribute"    "go.opentelemetry.io/otel/codes"    "go.opentelemetry.io/otel/trace"    "github.com/Arize-ai/openinference/go/openinference-semantic-conventions")func myFunc(ctx context.Context, input string) (string, error) {    tracer := otel.Tracer("my-app")    ctx, span := tracer.Start(ctx, "my-func",        trace.WithAttributes(            attribute.String(semconv.OpenInferenceSpanKind, semconv.SpanKindChain),            attribute.String(semconv.InputValue, input),        ),    )    defer span.End()    output := "output"    span.SetAttributes(attribute.String(semconv.OutputValue, output))    span.SetStatus(codes.Ok, "")    return output, nil}

This entire function will appear as a Span in Phoenix. Input and output attributes in Phoenix will be set automatically based on my_func’s parameters and return. The status attribute will also be set automatically.

2. As a with clause to trace specific code blocks#

with tracer.start_as_current_span(    "my-span-name",    openinference_span_kind="chain",) as span:    span.set_input("input")    span.set_output("output")    span.set_status(Status(StatusCode.OK))

import { withSpan } from "@arizeai/phoenix-otel";import { trace } from "@arizeai/phoenix-otel";await withSpan(  async () => {    const span = trace.getActiveSpan();    if (span) {      span.setAttributes({        "input.value": "input",        "output.value": "output",      });    }  },  {    name: "my-span-name",    kind: "CHAIN",  });

Go does not have a with statement; use tracer.Start + defer span.End() to scope a span to a block:

import (    "go.opentelemetry.io/otel"    "go.opentelemetry.io/otel/attribute"    "go.opentelemetry.io/otel/codes"    "go.opentelemetry.io/otel/trace"    "github.com/Arize-ai/openinference/go/openinference-semantic-conventions")tracer := otel.Tracer("my-app")ctx, span := tracer.Start(ctx, "my-span-name",    trace.WithAttributes(        attribute.String(semconv.OpenInferenceSpanKind, semconv.SpanKindChain),    ),)defer span.End()span.SetAttributes(    attribute.String(semconv.InputValue, "input"),    attribute.String(semconv.OutputValue, "output"),)span.SetStatus(codes.Ok, "")

The code within this clause will be captured as a Span in Phoenix. Here the input, output, and status must be set manually.

This approach is useful when you need only a portion of a method to be captured as a Span.

OpenInference Span Kinds#

OpenInference Span Kinds denote the possible types of spans you might capture, and will be rendered different in the Phoenix UI.

The openinference.span.kind attribute is required for all OpenInference spans and identifies the type of operation being traced. The span kind provides a hint to the tracing backend as to how the trace should be assembled. Valid values include:

Span Kind	Description
LLM	A span that represents a call to a Large Language Model (LLM). For example, an LLM span could be used to represent a call to OpenAI or Llama for chat completions or text generation.
EMBEDDING	A span that represents a call to an LLM or embedding service for generating embeddings. For example, an Embedding span could be used to represent a call to OpenAI to get an ada embedding for retrieval.
CHAIN	A span that represents a starting point or a link between different LLM application steps. For example, a Chain span could be used to represent the beginning of a request to an LLM application or the glue code that passes context from a retriever to an LLM call.
RETRIEVER	A span that represents a data retrieval step. For example, a Retriever span could be used to represent a call to a vector store or a database to fetch documents or information.
RERANKER	A span that represents the reranking of a set of input documents. For example, a cross-encoder may be used to compute the input documents’ relevance scores with respect to a user query, and the top K documents with the highest scores are then returned by the Reranker.
TOOL	A span that represents a call to an external tool such as a calculator, weather API, or any function execution that is invoked by an LLM or agent.
AGENT	A span that encompasses calls to LLMs and Tools. An agent describes a reasoning block that acts on tools using the guidance of an LLM.
GUARDRAIL	A span that represents calls to a component to protect against jailbreak user input prompts by taking action to modify or reject an LLM’s response if it contains undesirable content. For example, a Guardrail span could involve checking if an LLM’s output response contains inappropriate language, via a custom or external guardrail library, and then amending the LLM response to remove references to the inappropriate language.
EVALUATOR	A span that represents a call to a function or process performing an evaluation of the language model’s outputs. Examples include assessing the relevance, correctness, or helpfulness of the language model’s answers.
DECISION	Experimental. A span that represents a call to a decision model, which scores or selects among candidate options provided in the request rather than generating free-form text. Support is still being built out and no decision-model provider is fully supported yet.

Chains#

Using Context Managers#

with tracer.start_as_current_span(    "chain-span-with-plain-text-io",    openinference_span_kind="chain",) as span:    span.set_input("input")    span.set_output("output")    span.set_status(Status(StatusCode.OK))

Using Decorators#

@tracer.chaindef decorated_chain_with_plain_text_output(input: str) -> str:    return "output"decorated_chain_with_plain_text_output("input")

Using JSON Output

@tracer.chaindef decorated_chain_with_json_output(input: str) -> Dict[str, Any]:    return {"output": "output"}decorated_chain_with_json_output("input")

Overriding Span Name

@tracer.chain(name="decorated-chain-with-overridden-name")def this_name_should_be_overridden(input: str) -> Dict[str, Any]:    return {"output": "output"}this_name_should_be_overridden("input")

Using Wrappers#

import { withSpan } from "@arizeai/phoenix-otel";import { trace } from "@arizeai/phoenix-otel";await withSpan(  async () => {    const span = trace.getActiveSpan();    if (span) {      span.setAttributes({        "input.value": "input",        "output.value": "output",      });    }  },  {    name: "chain-span-with-plain-text-io",    kind: "CHAIN",  });

Using Function Wrappers#

import { traceChain } from "@arizeai/phoenix-otel";const decoratedChainWithPlainTextOutput = traceChain(  (input: string): string => {    return "output";  },  { name: "decorated-chain-with-plain-text-output" });decoratedChainWithPlainTextOutput("input");

Using JSON Serializable Output

import { traceChain } from "@arizeai/phoenix-otel";const decoratedChainWithJsonOutput = traceChain(  (input: string): Record<string, any> => {    return { output: "output" };  },  { name: "decorated-chain-with-json-output" });decoratedChainWithJsonOutput("input");

Overriding Span Name

import { traceChain } from "@arizeai/phoenix-otel";const thisNameShouldBeOverridden = traceChain(  (input: string): Record<string, any> => {    return { output: "output" };  },  { name: "decorated-chain-with-overridden-name" });thisNameShouldBeOverridden("input");

Set the OpenInference span kind to CHAIN on any span that represents a step or glue point in your application. Inputs and outputs go on the same span via the semconv constants:

import (    "go.opentelemetry.io/otel"    "go.opentelemetry.io/otel/attribute"    "go.opentelemetry.io/otel/trace"    "github.com/Arize-ai/openinference/go/openinference-semantic-conventions")tracer := otel.Tracer("my-app")ctx, span := tracer.Start(ctx, "my-chain",    trace.WithAttributes(        attribute.String(semconv.OpenInferenceSpanKind, semconv.SpanKindChain),        attribute.String(semconv.InputValue, "input"),    ),)defer span.End()// ... do work ...span.SetAttributes(attribute.String(semconv.OutputValue, "output"))

Agents#

Using Context Managers#

with tracer.start_as_current_span(    "agent-span-with-plain-text-io",    openinference_span_kind="agent",) as span:    span.set_input("input")    span.set_output("output")    span.set_status(Status(StatusCode.OK))

Using Decorators#

@tracer.agentdef decorated_agent(input: str) -> str:    return "output"decorated_agent("input")

Using Function Wrappers#

import { withSpan } from "@arizeai/phoenix-otel";import { trace } from "@arizeai/phoenix-otel";await withSpan(  async () => {    const span = trace.getActiveSpan();    if (span) {      span.setAttributes({        "input.value": "input",        "output.value": "output",      });    }  },  {    name: "agent-span-with-plain-text-io",    kind: "AGENT",  });

Using Function Wrappers#

import { traceAgent } from "@arizeai/phoenix-otel";const decoratedAgent = traceAgent(  (input: string): string => {    return "output";  },  { name: "decorated-agent" });decoratedAgent("input");

Use the AGENT span kind for spans that wrap an LLM-driven reasoning loop. Set agent.name to identify the agent across calls:

import (    "go.opentelemetry.io/otel"    "go.opentelemetry.io/otel/attribute"    "go.opentelemetry.io/otel/trace"    "github.com/Arize-ai/openinference/go/openinference-semantic-conventions")tracer := otel.Tracer("my-app")ctx, span := tracer.Start(ctx, "my-agent",    trace.WithAttributes(        attribute.String(semconv.OpenInferenceSpanKind, semconv.SpanKindAgent),        attribute.String(semconv.AgentName, "research-agent"),        attribute.String(semconv.InputValue, userQuery),    ),)defer span.End()// ... run agent loop, calling tools and LLMs (those produce child spans) ...span.SetAttributes(attribute.String(semconv.OutputValue, finalAnswer))

Tools#

Using Context Managers#

with tracer.start_as_current_span(    "tool-span",    openinference_span_kind="tool",) as span:    span.set_input("input")    span.set_output("output")    span.set_tool(        name="tool-name",        description="tool-description",        parameters={"input": "input"},    )    span.set_status(Status(StatusCode.OK))

Using Decorators#

@tracer.tooldef decorated_tool(input1: str, input2: int) -> None:    """    tool-description    """decorated_tool("input1", 1)

Overriding Tool Name

@tracer.tool(    name="decorated-tool-with-overridden-name",    description="overridden-tool-description",)def this_tool_name_should_be_overridden(input1: str, input2: int) -> None:    """    this tool description should be overridden    """this_tool_name_should_be_overridden("input1", 1)

Using Function Wrappers#

import { traceTool } from "@arizeai/phoenix-otel";/** * tool-description */const decoratedTool = traceTool(  (input1: string, input2: number): void => {    // Tool implementation  },  { name: "decorated-tool" });decoratedTool("input1", 1);

Overriding Tool Name

import { traceTool } from "@arizeai/phoenix-otel";/** * this tool description should be overridden */const thisToolNameShouldBeOverridden = traceTool(  (input1: string, input2: number): void => {    // Tool implementation  },  {    name: "decorated-tool-with-overridden-name",    description: "overridden-tool-description",  });thisToolNameShouldBeOverridden("input1", 1);

Use the TOOL span kind for spans that wrap calls to external functions, APIs, or any tool the agent invokes. The tool name and inputs are conventionally attached to the same span:

import (    "encoding/json"    "go.opentelemetry.io/otel"    "go.opentelemetry.io/otel/attribute"    "go.opentelemetry.io/otel/trace"    "github.com/Arize-ai/openinference/go/openinference-semantic-conventions")tracer := otel.Tracer("my-app")toolArgs := map[string]any{"city": "Johannesburg"}toolArgsJSON, _ := json.Marshal(toolArgs)ctx, span := tracer.Start(ctx, "lookup_weather",    trace.WithAttributes(        attribute.String(semconv.OpenInferenceSpanKind, semconv.SpanKindTool),        attribute.String(semconv.InputValue, string(toolArgsJSON)),        attribute.String(semconv.InputMimeType, semconv.MimeTypeJSON),    ),)defer span.End()result := lookupWeather(toolArgs["city"].(string))span.SetAttributes(attribute.String(semconv.OutputValue, result))

LLMs#

Like other span kinds, LLM spans can be instrumented either via a context manager or via a decorator pattern. It’s also possible to directly patch client methods.

While this guide uses the OpenAI Python client for illustration, in practice, you should use the OpenInference auto-instrumentors for OpenAI whenever possible and resort to manual instrumentation for LLM spans only as a last resort.

To run the snippets in this section, set your OPENAI_API_KEY environment variable.

Context Manager#

from openai import OpenAIfrom opentelemetry.trace import Status, StatusCodeopenai_client = OpenAI()messages = [{"role": "user", "content": "Hello, world!"}]with tracer.start_as_current_span("llm_span", openinference_span_kind="llm") as span:    span.set_input(messages)    try:        response = openai_client.chat.completions.create(            model="gpt-4",            messages=messages,        )    except Exception as error:        span.record_exception(error)        span.set_status(Status(StatusCode.ERROR))    else:        span.set_output(response)        span.set_status(Status(StatusCode.OK))

Decorator#

from typing import Listfrom openai import OpenAIfrom openai.types.chat import ChatCompletionMessageParamopenai_client = OpenAI()@tracer.llmdef invoke_llm(    messages: List[ChatCompletionMessageParam],) -> str:    response = openai_client.chat.completions.create(        model="gpt-4o",        messages=messages,    )    message = response.choices[0].message    return message.content or ""invoke_llm([{"role": "user", "content": "Hello, world!"}])

This decorator pattern above works for sync functions, async coroutine functions, sync generator functions, and async generator functions. Here’s an example with an async generator.

from typing import AsyncGenerator, Listfrom openai import AsyncOpenAIfrom openai.types.chat import ChatCompletionMessageParamopenai_async_client = AsyncOpenAI()@tracer.llmasync def stream_llm_responses(    messages: List[ChatCompletionMessageParam],) -> AsyncGenerator[str, None]:    stream = await openai_async_client.chat.completions.create(        model="gpt-4o",        messages=messages,        stream=True,    )    async for chunk in stream:        if chunk.choices[0].delta.content:            yield chunk.choices[0].delta.content# invoke inside of an async contextasync for token in stream_llm_responses([{"role": "user", "content": "Hello, world!"}]):    print(token, end="")

Method Patch#

It’s also possible to directly patch methods on a client. This is useful if you want to transparently use the client in your application with instrumentation logic localized in one place.

from openai import OpenAIopenai_client = OpenAI()# patch the create methodwrapper = tracer.llmopenai_client.chat.completions.create = wrapper(openai_client.chat.completions.create)# invoke the patched method normallyopenai_client.chat.completions.create(    model="gpt-4o",    messages=[{"role": "user", "content": "Hello, world!"}],)

The snippets above produce LLM spans with input and output values, but don’t offer rich UI for messages, tools, invocation parameters, etc. In order to manually instrument LLM spans with these features, users can define their own functions to wrangle the input and output of their LLM calls into OpenInference format. These richer LLM attribute helpers are not re-exported from phoenix.otel — install openinference-instrumentation to access them:

get_llm_attributes
get_input_attributes
get_output_attributes

pip install openinference-instrumentation

For OpenAI, these functions might look like this:

from typing import Any, Dict, List, Optional, Unionfrom openai.types.chat import (    ChatCompletion,    ChatCompletionMessage,    ChatCompletionMessageParam,    ChatCompletionToolParam,)from opentelemetry.util.types import AttributeValueimport openinference.instrumentation as oifrom openinference.instrumentation import (    get_input_attributes,    get_llm_attributes,    get_output_attributes,)def process_input(    messages: List[ChatCompletionMessageParam],    model: str,    temperature: Optional[float] = None,    tools: Optional[List[ChatCompletionToolParam]] = None,    **kwargs: Any,) -> Dict[str, AttributeValue]:    oi_messages = [convert_openai_message_to_oi_message(message) for message in messages]    oi_tools = [convert_openai_tool_param_to_oi_tool(tool) for tool in tools or []]    return {        **get_input_attributes(            {                "messages": messages,                "model": model,                "temperature": temperature,                "tools": tools,                **kwargs,            }        ),        **get_llm_attributes(            provider="openai",            system="openai",            model_name=model,            input_messages=oi_messages,            invocation_parameters={"temperature": temperature},            tools=oi_tools,        ),    }def convert_openai_message_to_oi_message(    message_param: Union[ChatCompletionMessageParam, ChatCompletionMessage],) -> oi.Message:    if isinstance(message_param, ChatCompletionMessage):        role: str = message_param.role        oi_message = oi.Message(role=role)        if isinstance(content := message_param.content, str):            oi_message["content"] = content        if message_param.tool_calls is not None:            oi_tool_calls: List[oi.ToolCall] = []            for tool_call in message_param.tool_calls:                function = tool_call.function                oi_tool_calls.append(                    oi.ToolCall(                        id=tool_call.id,                        function=oi.ToolCallFunction(                            name=function.name,                            arguments=function.arguments,                        ),                    )                )            oi_message["tool_calls"] = oi_tool_calls        return oi_message    role = message_param["role"]    assert isinstance(message_param["content"], str)    content = message_param["content"]    return oi.Message(role=role, content=content)def convert_openai_tool_param_to_oi_tool(tool_param: ChatCompletionToolParam) -> oi.Tool:    assert tool_param["type"] == "function"    return oi.Tool(json_schema=dict(tool_param))def process_output(response: ChatCompletion) -> Dict[str, AttributeValue]:    message = response.choices[0].message    role = message.role    oi_message = oi.Message(role=role)    if isinstance(message.content, str):        oi_message["content"] = message.content    if isinstance(message.tool_calls, list):        oi_tool_calls: List[oi.ToolCall] = []        for tool_call in message.tool_calls:            tool_call_id = tool_call.id            function_name = tool_call.function.name            function_arguments = tool_call.function.arguments            oi_tool_calls.append(                oi.ToolCall(                    id=tool_call_id,                    function=oi.ToolCallFunction(                        name=function_name,                        arguments=function_arguments,                    ),                )            )        oi_message["tool_calls"] = oi_tool_calls    output_messages = [oi_message]    token_usage = response.usage    oi_token_count: Optional[oi.TokenCount] = None    if token_usage is not None:        prompt_tokens = token_usage.prompt_tokens        completion_tokens = token_usage.completion_tokens        oi_token_count = oi.TokenCount(            prompt=prompt_tokens,            completion=completion_tokens,        )    return {        **get_llm_attributes(            output_messages=output_messages,            token_count=oi_token_count,        ),        **get_output_attributes(response),    }

Context Manager#

When using a context manager to create LLM spans, these functions can be used to wrangle inputs and outputs.

import jsonfrom openai import OpenAIfrom openai.types.chat import (    ChatCompletionMessage,    ChatCompletionMessageParam,    ChatCompletionToolMessageParam,    ChatCompletionToolParam,    ChatCompletionUserMessageParam,)from opentelemetry.trace import Status, StatusCodeopenai_client = OpenAI()@tracer.tooldef get_weather(city: str) -> str:    # make an call to a weather API here    return "sunny"messages: List[Union[ChatCompletionMessage, ChatCompletionMessageParam]] = [    ChatCompletionUserMessageParam(        role="user",        content="What's the weather like in San Francisco?",    )]temperature = 0.5invocation_parameters = {"temperature": temperature}tools: List[ChatCompletionToolParam] = [    {        "type": "function",        "function": {            "name": "get_weather",            "description": "finds the weather for a given city",            "parameters": {                "type": "object",                "properties": {                    "city": {                        "type": "string",                        "description": "The city to find the weather for, e.g. 'London'",                    }                },                "required": ["city"],            },        },    },]with tracer.start_as_current_span(    "llm_tool_call",    attributes=process_input(        messages=messages,        invocation_parameters={"temperature": temperature},        model="gpt-4",    ),    openinference_span_kind="llm",) as span:    try:        response = openai_client.chat.completions.create(            model="gpt-4o",            messages=messages,            temperature=temperature,            tools=tools,        )    except Exception as error:        span.record_exception(error)        span.set_status(Status(StatusCode.ERROR))    else:        span.set_attributes(process_output(response))        span.set_status(Status(StatusCode.OK))output_message = response.choices[0].messagetool_calls = output_message.tool_callsassert tool_calls and len(tool_calls) == 1tool_call = tool_calls[0]city = json.loads(tool_call.function.arguments)["city"]weather = get_weather(city)messages.append(output_message)messages.append(    ChatCompletionToolMessageParam(        content=weather,        role="tool",        tool_call_id=tool_call.id,    ))with tracer.start_as_current_span(    "tool_call_response",    attributes=process_input(        messages=messages,        invocation_parameters={"temperature": temperature},        model="gpt-4",    ),    openinference_span_kind="llm",) as span:    try:        response = openai_client.chat.completions.create(            model="gpt-4o",            messages=messages,            temperature=temperature,        )    except Exception as error:        span.record_exception(error)        span.set_status(Status(StatusCode.ERROR))    else:        span.set_attributes(process_output(response))        span.set_status(Status(StatusCode.OK))

Decorator#

When using the tracer.llm decorator, these functions are passed via the process_input and process_output parameters and should satisfy the following:

The input signature of process_input should exactly match the input signature of the decorated function.
The input signature of process_output has a single argument, the output of the decorated function. This argument accepts the returned value when the decorated function is a sync or async function, or a list of yielded values when the decorated function is a sync or async generator function.
Both process_input and process_output should output a dictionary mapping attribute names to values.

from openai import NOT_GIVEN, OpenAIfrom openai.types.chat import ChatCompletionopenai_client = OpenAI()@tracer.llm(    process_input=process_input,    process_output=process_output,)def invoke_llm(    messages: List[ChatCompletionMessageParam],    model: str,    temperature: Optional[float] = None,    tools: Optional[List[ChatCompletionToolParam]] = None,) -> ChatCompletion:    response: ChatCompletion = openai_client.chat.completions.create(        messages=messages,        model=model,        tools=tools or NOT_GIVEN,        temperature=temperature,    )    return responseinvoke_llm(    messages=[{"role": "user", "content": "Hello, world!"}],    temperature=0.5,    model="gpt-4",)

When decorating a generator function, process_output should accept a single argument, a list of the values yielded by the decorated function.

from typing import Dict, List, Optionalfrom openai.types.chat import ChatCompletionChunkfrom opentelemetry.util.types import AttributeValueimport openinference.instrumentation as oifrom openinference.instrumentation import (    get_llm_attributes,    get_output_attributes,)def process_generator_output(    outputs: List[ChatCompletionChunk],) -> Dict[str, AttributeValue]:    role: Optional[str] = None    content = ""    oi_token_count = oi.TokenCount()    for chunk in outputs:        if choices := chunk.choices:            assert len(choices) == 1            delta = choices[0].delta            if isinstance(delta.content, str):                content += delta.content            if isinstance(delta.role, str):                role = delta.role        if (usage := chunk.usage) is not None:            if (prompt_tokens := usage.prompt_tokens) is not None:                oi_token_count["prompt"] = prompt_tokens            if (completion_tokens := usage.completion_tokens) is not None:                oi_token_count["completion"] = completion_tokens    oi_messages = []    if role and content:        oi_messages.append(oi.Message(role=role, content=content))    return {        **get_llm_attributes(            output_messages=oi_messages,            token_count=oi_token_count,        ),        **get_output_attributes(content),    }

Then the decoration is the same as before.

from typing import AsyncGeneratorfrom openai import AsyncOpenAIfrom openai.types.chat import ChatCompletionChunkopenai_async_client = AsyncOpenAI()@tracer.llm(    process_input=process_input,  # same as before    process_output=process_generator_output,)async def stream_llm_response(    messages: List[ChatCompletionMessageParam],    model: str,    temperature: Optional[float] = None,) -> AsyncGenerator[ChatCompletionChunk, None]:    async for chunk in await openai_async_client.chat.completions.create(        messages=messages,        model=model,        temperature=temperature,        stream=True,    ):        yield chunkasync for chunk in stream_llm_response(    messages=[{"role": "user", "content": "Hello, world!"}],    temperature=0.5,    model="gpt-4",):    print(chunk)

Method Patch#

As before, it’s possible to directly patch the method on the client. Just ensure that the input signatures of process_input and the patched method match.

from openai import OpenAIfrom openai.types.chat import ChatCompletionMessageParamopenai_client = OpenAI()# patch the create methodwrapper = tracer.llm(    process_input=process_input,    process_output=process_output,)openai_client.chat.completions.create = wrapper(openai_client.chat.completions.create)# invoke the patched method normallyopenai_client.chat.completions.create(    model="gpt-4o",    messages=[{"role": "user", "content": "Hello, world!"}],)

Function Wrapper#

import { withSpan } from "@arizeai/phoenix-otel";import OpenAI from "openai";const openaiClient = new OpenAI();const invokeLLM = withSpan(  async (    messages: Array<{ role: string; content: string }>  ): Promise<string> => {    const response = await openaiClient.chat.completions.create({      model: "gpt-4o",      messages: messages,    });    const message = response.choices[0].message;    return message.content || "";  },  {    name: "invoke-llm",    kind: "LLM",  });await invokeLLM([{ role: "user", content: "Hello, world!" }]);

The snippets above produce LLM spans with input and output values, but don’t offer rich UI for messages, tools, invocation parameters, etc. In order to manually instrument LLM spans with these features, users can use helper functions from @arizeai/phoenix-otel that produce valid OpenInference attributes for LLM spans:

getLLMAttributes
defaultProcessInput
defaultProcessOutput

For OpenAI, these functions might look like this:

import {  getLLMAttributes,  defaultProcessInput,  defaultProcessOutput,} from "@arizeai/phoenix-otel";import OpenAI from "openai";interface ChatCompletionMessageParam {  role: string;  content: string;}interface ChatCompletionToolParam {  type: string;  function: {    name: string;    description: string;    parameters: Record<string, unknown>;  };}function processInput(  messages: ChatCompletionMessageParam[],  model: string,  temperature?: number,  tools?: ChatCompletionToolParam[]) {  const inputAttrs = defaultProcessInput({    messages,    model,    temperature,    tools,  });  const llmAttrs = getLLMAttributes({    provider: "openai",    system: "openai",    modelName: model,    inputMessages: messages.map((msg) => ({      role: msg.role,      content: msg.content,    })),    invocationParameters: { temperature },    tools: tools?.map((tool) => ({ jsonSchema: tool })),  });  return { ...inputAttrs, ...llmAttrs };}function processOutput(response: OpenAI.Chat.Completions.ChatCompletion) {  const message = response.choices[0].message;  const outputAttrs = defaultProcessOutput(response);  const llmAttrs = getLLMAttributes({    outputMessages: [      {        role: message.role,        content: typeof message.content === "string" ? message.content : "",      },    ],    tokenCount: response.usage      ? {          prompt: response.usage.prompt_tokens,          completion: response.usage.completion_tokens,          total: response.usage.total_tokens,        }      : undefined,  });  return { ...outputAttrs, ...llmAttrs };}

Function Wrapper#

When using withSpan to wrap functions, you can pass processInput and processOutput functions as options. These should satisfy the following:

The input signature of processInput should exactly match the input signature of the wrapped function.
The input signature of processOutput has a single argument, the output of the wrapped function. This argument accepts the returned value when the wrapped function is a sync or async function.
Both processInput and processOutput should output a dictionary mapping attribute names to values.

import { withSpan } from "@arizeai/phoenix-otel";import OpenAI from "openai";const openaiClient = new OpenAI();const invokeLLM = withSpan(  async (    messages: ChatCompletionMessageParam[],    model: string,    temperature?: number,    tools?: ChatCompletionToolParam[]  ): Promise<OpenAI.Chat.Completions.ChatCompletion> => {    const response = await openaiClient.chat.completions.create({      messages: messages,      model: model,      tools: tools,      temperature: temperature,    });    return response;  },  {    name: "invoke-llm",    kind: "LLM",    processInput: (messages, model, temperature, tools) =>      processInput(messages, model, temperature, tools),    processOutput: (response) => processOutput(response),  });await invokeLLM([{ role: "user", content: "Hello, world!" }], "gpt-4", 0.5);

For SDKs with OpenInference Go auto-instrumentation, see OpenAI Go SDK and Anthropic SDK Go — no manual span code required. For SDKs without auto-instrumentation, attach model and token attributes directly using semconv:

import (    "go.opentelemetry.io/otel"    "go.opentelemetry.io/otel/attribute"    "go.opentelemetry.io/otel/trace"    "github.com/Arize-ai/openinference/go/openinference-semantic-conventions")tracer := otel.Tracer("my-app")ctx, span := tracer.Start(ctx, "invoke_llm",    trace.WithAttributes(        attribute.String(semconv.OpenInferenceSpanKind, semconv.SpanKindLLM),        attribute.String(semconv.LLMProvider, semconv.LLMProviderOpenAI),        attribute.String(semconv.LLMSystem, semconv.LLMSystemOpenAI),        attribute.String(semconv.LLMModelName, "gpt-4"),        attribute.String(semconv.InputValue, "Hello, world!"),    ),)defer span.End()// ... make the LLM call ...span.SetAttributes(    attribute.String(semconv.OutputValue, completionText),    attribute.Int(semconv.LLMTokenCountPrompt, promptTokens),    attribute.Int(semconv.LLMTokenCountCompletion, completionTokens),    attribute.Int(semconv.LLMTokenCountTotal, totalTokens),)

Additional Features#

The OpenInference Tracer shown above respects context Managers for Suppressing Tracing & Adding Metadata

Suppress Tracing#

from phoenix.otel import suppress_tracingwith suppress_tracing():    # this trace will not be recorded    with tracer.start_as_current_span(        "THIS-SPAN-SHOULD-NOT-BE-TRACED",        openinference_span_kind="chain",    ) as span:        span.set_input("input")        span.set_output("output")        span.set_status(Status(StatusCode.OK))

import { withSpan } from "@arizeai/phoenix-otel";import { trace, context, suppressTracing } from "@arizeai/phoenix-otel";await context.with(suppressTracing(context.active()), async () => {  // this trace will not be recorded  await withSpan(    async () => {      const span = trace.getActiveSpan();      if (span) {        span.setAttributes({          "input.value": "input",          "output.value": "output",        });      }    },    {      name: "THIS-SPAN-SHOULD-NOT-BE-TRACED",      kind: "CHAIN",    }  );});

Use instrumentation.WithSuppression to mark a context such that any OpenInference-aware Go instrumentor (e.g. the OpenAI and Anthropic Go transports / middlewares) skips span emission for calls descended from it. Typical use: evaluator or grader code that itself calls an LLM but should not appear in the customer’s product trace.

import (    "github.com/Arize-ai/openinference/go/openinference-instrumentation")ctx := instrumentation.WithSuppression(ctx)resp, _ := client.CreateChatCompletion(ctx, req)  // no LLM span emitted

For manual spans you author yourself, check the context before calling tracer.Start if you want to honor suppression in your own code.

Using Context Attributes#

from phoenix.otel import using_attributeswith using_attributes(session_id="123"):    # this trace has session id "123"    with tracer.start_as_current_span(        "chain-span-with-context-attributes",        openinference_span_kind="chain",    ) as span:        span.set_input("input")        span.set_output("output")        span.set_status(Status(StatusCode.OK))

import { context } from "@opentelemetry/api";import { setSession } from "@arizeai/phoenix-otel";import { withSpan } from "@arizeai/phoenix-otel";import { trace } from "@arizeai/phoenix-otel";await context.with(  setSession(context.active(), { sessionId: "123" }),  async () => {    // this trace has session id "123"    await withSpan(      async () => {        const span = trace.getActiveSpan();        if (span) {          span.setAttributes({            "input.value": "input",            "output.value": "output",          });        }      },      {        name: "chain-span-with-context-attributes",        kind: "CHAIN",      }    );  });

Use the helpers in github.com/Arize-ai/openinference/go/openinference-instrumentation to store session, user, metadata, and tags on context.Context. Auto-instrumentors (OpenAI Go, Anthropic Go) automatically apply these to every LLM span descended from the context:

import (    "encoding/json"    "github.com/Arize-ai/openinference/go/openinference-instrumentation")metadata := map[string]any{"key-1": "value_1", "key-2": "value_2"}metadataJSON, _ := json.Marshal(metadata)ctx = instrumentation.WithSession(ctx, "my-session-id")ctx = instrumentation.WithUser(ctx, "my-user-id")ctx = instrumentation.WithMetadata(ctx, string(metadataJSON))ctx = instrumentation.WithTags(ctx, "tag_1", "tag_2")// All LLM spans descended from ctx now carry session.id, user.id,// metadata, and tag.tags — no per-span code needed.resp, _ := client.CreateChatCompletion(ctx, req)

For manual spans you author yourself, set the same attributes directly via the semconv keys (SessionID, UserID, Metadata, TagTags) after tracer.Start.

Adding Images to your Traces#

OpenInference includes message types that can be useful in composing text and image or other file inputs and outputs:

import openinference.instrumentation as oiimage_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"text = "describe the weather in this image"content = [        {"type": "text", "text": text},        {            "type": "image_url",            "image_url": {"url": image_url, "detail": "low"},        },    ]image = oi.Image(url=image_url)contents = [    oi.TextMessageContent(        type="text",        text=text,    ),    oi.ImageMessageContent(        type="image",        image=image,    ),]messages = [    oi.Message(        role="user",        contents=contents,    )]with tracer.start_as_current_span(    "my-span-name",    openinference_span_kind="llm",    attributes=oi.get_llm_attributes(input_messages=messages)) as span:    span.set_input(text)    # Call your LLM here    response = "This is a test response"    span.set_output(response)    print(response.content)

import { withSpan, getLLMAttributes } from "@arizeai/phoenix-otel";import { trace } from "@arizeai/phoenix-otel";const imageUrl =  "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg";const text = "describe the weather in this image";const messages = [  {    role: "user",    contents: [      {        type: "text",        text: text,      },      {        type: "image",        image: {          url: imageUrl,        },      },    ],  },];await withSpan(  async () => {    const span = trace.getActiveSpan();    if (span) {      span.setAttributes(        getLLMAttributes({          inputMessages: messages,        })      );      // Call your LLM here      const response = "This is a test response";      span.setAttributes({        "input.value": text,        "output.value": response,      });      console.log(response);    }  },  {    name: "my-span-name",    kind: "LLM",  });

For SDKs with OpenInference Go auto-instrumentation (OpenAI Go, Anthropic Go), multimodal content is captured automatically when you call the SDK with image-containing messages — no per-span code required.

For manual spans, attach image content using the indexed llm.input_messages.{i}.message.contents.{j}.* attributes from github.com/Arize-ai/openinference/go/openinference-semantic-conventions. The Python openinference.instrumentation message-builder helpers do not yet have a Go equivalent — refer to the OpenInference spec for the full multimodal attribute taxonomy.