Log Evaluation Results

Evaluations, which can be considered a form of automated annotation, are logged as annotations inside of Phoenix. Instead of coming from a “HUMAN” source, they are either “CODE” (aka heuristic) or “LLM” kinds.

An evaluation must have a name (e.g. “Q&A Correctness”) and its DataFrame must contain identifiers for the subject of evaluation, e.g. a span or a document (more on that below), and values under either the score, label, or explanation columns. An optional metadata column can also be provided.

Connect to Phoenix#

Initialize the Phoenix client to connect to your Phoenix instance:

from phoenix.client import Client# Initialize client - automatically reads from environment variables:# PHOENIX_ENDPOINT, and PHOENIX_API_KEY if authentication is enabledclient = Client()# Or explicitly configure for your Phoenix instance:# client = Client(base_url="https://your-phoenix-instance.com", api_key="your-api-key")

Span Evaluations#

A dataframe of span evaluations would look similar to the table below. It must contain span_id as an index or as a column. Once ingested, Phoenix uses the span_id to associate the evaluation with its target span.

The evaluations dataframe can be sent to Phoenix as follows.

Note that the name and kind of the evaluation can be supplied through the annotation_name annotator_kind parameters, or as columns with the same names in the dataframe.

from phoenix.client import ClientClient().spans.log_span_annotations_dataframe(    dataframe=qa_correctness_eval_df,    annotation_name="Q&A Correctness",    annotator_kind="LLM",)

Document Evaluations#

A dataframe of document evaluations would look something like the table below. It must contain span_id and document_position as either indices or columns. document_position is the document’s (zero-based) index in the span’s list of retrieved documents. Once ingested, Phoenix uses the span_id and document_position to associate the evaluation with its target span and document.

The evaluations dataframe can be sent to Phoenix as follows. In this case we name it “Relevance”.

from phoenix.client import ClientClient().spans.log_document_annotations_dataframe(    dataframe=retrieval_relevance_eval_df,    annotation_name="Relevance",    annotator_kind="LLM",)

Logging Multiple Evaluation DataFrames#

Multiple sets of Evaluations can be logged using separate function calls with the new client.

client.spans.log_span_annotations_dataframe(    dataframe=qa_correctness_eval_df,    annotation_name="Q&A Correctness",    annotator_kind="LLM",)client.spans.log_document_annotations_dataframe(    dataframe=retrieval_relevance_eval_df,    annotation_name="Relevance",    annotator_kind="LLM",)client.spans.log_span_annotations_dataframe(    dataframe=faithfulness_eval_df,    annotation_name="Faithfulness",    annotator_kind="LLM",)# ... continue with additional evaluations as needed

Or, if you specify the annotation_name and annotator_kind as columns, you can vertically concatenate the dataframes and upload them all at once.

import pandas as pdqa_correctness_eval_df["annotation_name"] = "QA Correctness"qa_correctness_eval_df["annotator_kind"] = "LLM"faithfulness_eval_df["annotation_name"] = "Faithfulness"faithfulness_eval_df["annotator_kind"] = "LLM"annotations_df = pd.concat([qa_correctness_eval_df, faithfulness_eval_df], ignore_index=True)px_client.spans.log_span_annotations_dataframe(dataframe=annotations_df)