LiteLLM Evals - Phoenix

Using LiteLLM with Phoenix Evals#

Info

Requires litellm>=1.0.3

pip install "arize-phoenix-evals>=3" litellm

Create an LLM instance with the LiteLLM provider:

from phoenix.evals import LLMllm = LLM(provider="litellm", model="gpt-4o")

LiteLLM supports 100+ providers. Set the appropriate environment variables for your provider before creating the LLM. For provider-specific configuration, see LiteLLM provider params.

Ollama example#

import osfrom phoenix.evals import LLMos.environ["OLLAMA_API_BASE"] = "http://localhost:11434"llm = LLM(provider="litellm", model="ollama/llama3")

Using with evaluators#

from phoenix.evals import LLM, evaluate_dataframefrom phoenix.evals.metrics import FaithfulnessEvaluatorllm = LLM(provider="litellm", model="ollama/llama3")evaluator = FaithfulnessEvaluator(llm=llm)results_df = evaluate_dataframe(dataframe=df, evaluators=[evaluator])