LiteLLM Tracing

LiteLLM allows developers to call all LLM APIs using the openAI format. LiteLLM Proxy is a proxy server to call 100+ LLMs in OpenAI format. Both are supported by this auto-instrumentation.

Any calls made to the following functions will be automatically captured by this integration:

• completion()

• acompletion()

• completion_with_retries()

• embedding()

• aembedding()

• image_generation()

• aimage_generation()

Install#

pip install openinference-instrumentation-litellm "litellm<1.82.7"

Setup#

Connect your application to Phoenix with the register function:

from phoenix.otel import register# configure the Phoenix tracertracer_provider = register(project_name="my-llm-app", # Default is 'default'auto_instrument=True # Auto-instrument your app based on installed OI dependencies)

Add any API keys needed by the models you are using with LiteLLM.

import osos.environ["OPENAI_API_KEY"] = "PASTE_YOUR_API_KEY_HERE"

Run LiteLLM#

You can now use LiteLLM as normal and calls will be traces in Phoenix.

import litellmcompletion_response = litellm.completion(model="gpt-3.5-turbo",                   messages=[{"content": "What's the capital of China?", "role": "user"}])print(completion_response)

Observe#

Traces should now be visible in Phoenix!

Resources#

• OpenInference Instrumentation