Portkey Tracing

Phoenix provides seamless integration with Portkey, the AI Gateway and observability platform that routes to 200+ LLMs with enterprise-grade features including guardrails, caching, and load balancing.

Install#

pip install openinference-instrumentation-portkey portkey-ai

Setup#

Connect your application to Phoenix with the register function:

from phoenix.otel import register# configure the Phoenix tracertracer_provider = register(project_name="my-portkey-app", # Default is 'default'auto_instrument=True # Auto-instrument your app based on installed OI dependencies)

Run Portkey#

By instrumenting Portkey, spans will be created whenever requests are made through the AI Gateway and will be sent to the Phoenix server for collection.

Basic Usage with OpenAI#

import osfrom openai import OpenAIfrom portkey_ai import PORTKEY_GATEWAY_URL, createHeaders# Set up your API keysos.environ["OPENAI_API_KEY"] = "your-openai-api-key"os.environ["PORTKEY_API_KEY"] = "your-portkey-api-key"  # Optional for self-hostedclient = OpenAI(    api_key=os.environ.get("OPENAI_API_KEY"),    base_url=PORTKEY_GATEWAY_URL,    default_headers=createHeaders(        provider="openai",        api_key=os.environ.get("PORTKEY_API_KEY")    ))response = client.chat.completions.create(    model="gpt-4o-mini",    messages=[{"role": "user", "content": "What is artificial intelligence?"}])print(response.choices[0].message.content)

Using Portkey SDK Directly#

from portkey_ai import Portkey# Initialize Portkey clientportkey = Portkey(    api_key="your-portkey-api-key",  # Optional for self-hosted    virtual_key="your-openai-virtual-key"  # Or use provider-specific virtual keys)response = portkey.chat.completions.create(    model="gpt-4o-mini",    messages=[{"role": "user", "content": "Explain machine learning"}])print(response.choices[0].message.content)

Observe#

Now that you have tracing setup, all requests through Portkey’s AI Gateway will be streamed to your running Phoenix instance for observability and evaluation. You’ll be able to see:

Request/Response Traces: Complete visibility into LLM interactions
Routing Decisions: Which provider was selected and why
Fallback Events: When and why fallbacks were triggered
Cache Performance: Hit/miss rates and response times
Cost Tracking: Token usage and costs across providers
Latency Metrics: Response times for each provider and route

Resources#

Phoenix OpenInference Instrumentation