Exact Match

Overview#

The exact_match evaluator is a simple code-based evaluator that checks if the output exactly equals the expected value. It performs a strict string comparison with no normalization.

When to Use#

Use the exact_match evaluator when you need to:

• Validate exact outputs - Check that responses match expected values character-for-character
• Evaluate classification tasks - Verify categorical outputs match expected labels
• Test deterministic outputs - Validate outputs that should be identical every time
• Quick sanity checks - Fast evaluation without LLM costs

Supported Levels#

Input Requirements#

The exact_match evaluator requires two inputs:

Important Notes#

• No normalization: The comparison is case-sensitive and whitespace-sensitive
• String comparison: Both inputs are compared as strings
• No partial matching: The entire string must match exactly

Output Interpretation#

The evaluator returns a Score object with the following properties:

Usage Examples#

from phoenix.evals.metrics import exact_match# Basic usage with matching field nameseval_input = {    "output": "Paris",    "expected": "Paris"}scores = exact_match.evaluate(eval_input)print(scores[0])# Score(name='exact_match', score=1.0, label=True, kind='code', ...)# Non-matching exampleeval_input = {    "output": "paris",  # lowercase    "expected": "Paris"  # uppercase}scores = exact_match.evaluate(eval_input)print(scores[0].score)  # 0.0 (case-sensitive comparison)

import { createEvaluator } from "@arizeai/phoenix-evals";const exactMatchEvaluator = createEvaluator(  (record: { output: string; expected: string }) => ({    score: record.output === record.expected ? 1 : 0,    label: record.output === record.expected ? "match" : "no_match",  }),  { name: "exact_match", kind: "CODE" });const result = await exactMatchEvaluator.evaluate({  output: "Paris",  expected: "Paris",});console.log(result); // { score: 1, label: "match" }

Implementing Case-Insensitive Matching#

If you need case-insensitive matching, normalize your inputs first:

from phoenix.evals.metrics import exact_matcheval_input = {    "output": "PARIS".lower(),    "expected": "paris"}scores = exact_match.evaluate(eval_input)print(scores[0].score)  # 1.0

Or create a custom evaluator with normalization:

from phoenix.evals.evaluators import Score, create_evaluator@create_evaluator(name="exact_match_normalized", kind="code")def exact_match_normalized(output: str, expected: str) -> Score:    """Case-insensitive exact match with whitespace normalization."""    normalized_output = output.strip().lower()    normalized_expected = expected.strip().lower()    correct = normalized_output == normalized_expected    return Score(score=float(correct))

Using with Phoenix#

Evaluating Traces#

Run evaluations on traces collected in Phoenix and log results as annotations:

• Evaluating Phoenix Traces

Running Experiments#

Use the exact_match evaluator in Phoenix experiments:

• Using Evaluators in Experiments

API Reference#

• Python: exact_match

Related#

• Matches Regex Evaluator - For pattern-based matching