Matches Regex - Phoenix

Overview#

The MatchesRegex evaluator is a code-based evaluator that checks if the output contains substrings matching a specified regular expression pattern. It’s useful for validating output format, detecting specific content patterns, or checking for required elements.

Note

This evaluator is only available as a built-in for Python. For TypeScript, see the usage example below showing how to create an equivalent evaluator using createEvaluator.

When to Use#

Use the MatchesRegex evaluator when you need to:

Validate output format - Check that responses follow expected patterns (URLs, emails, dates)
Detect specific content - Find phone numbers, IDs, or other structured data in outputs
Enforce formatting rules - Verify outputs contain required elements
Pattern-based quality checks - Check for presence of citations, code blocks, or other patterns

Info

This is a code-based evaluator using Python’s re module. For exact string matching, use exact_match instead.

Supported Levels#

Level	Supported	Notes
Span	Yes	Evaluate any span output against regex patterns.

Input Requirements#

The MatchesRegex evaluator requires one input:

Field	Type	Description
`output`	`string`	The text to evaluate against the regex pattern

Constructor Arguments#

Argument	Type	Description
`pattern`	`str` or `Pattern`	The regex pattern (string or compiled)
`name`	`str` (optional)	Custom evaluator name (default: “matches_regex”)
`include_explanation`	`bool` (optional)	Include match details in explanation (default: True)

Output Interpretation#

The evaluator returns a Score object with the following properties:

Property	Value	Description
`score`	`1.0` or `0.0`	1.0 if pattern matches, 0.0 if no match
`explanation`	`string`	Number of matches found or “no match” message
`kind`	`"code"`	Indicates this is a code-based evaluator
`direction`	`"maximize"`	Higher scores are better

Usage Examples#

import refrom phoenix.evals.metrics import MatchesRegex# Create evaluator with a URL detection patternurl_pattern = re.compile(r"https?://[^\s]+")contains_url = MatchesRegex(pattern=url_pattern)# Inspect the evaluator's requirementsprint(contains_url.describe())# Evaluate output with a URLeval_input = {"output": "Check out https://github.com/Arize-ai/phoenix!"}scores = contains_url.evaluate(eval_input)print(scores[0])# Score(name='matches_regex', score=1.0, explanation='There are 1 matches...', ...)# Evaluate output without a URLeval_input = {"output": "This text has no links."}scores = contains_url.evaluate(eval_input)print(scores[0].score)  # 0.0

Note

The MatchesRegex evaluator is not available as a built-in for TypeScript. You can create an equivalent code evaluator using createEvaluator:

import { createEvaluator } from "@arizeai/phoenix-evals";const urlPattern = /https?:\/\/[^\s]+/g;const matchesUrlEvaluator = createEvaluator(  (record: { output: string }) => {    const matches = record.output.match(urlPattern);    return {      score: matches ? 1 : 0,      explanation: matches        ? `Found ${matches.length} URL(s)`        : "No URLs found",    };  },  { name: "matches_url", kind: "CODE" });const result = await matchesUrlEvaluator.evaluate({  output: "Visit https://phoenix.arize.com for more info",});console.log(result); // { score: 1, explanation: "Found 1 URL(s)" }

Common Pattern Examples#

import refrom phoenix.evals.metrics import MatchesRegex# Email detectionemail_pattern = re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")contains_email = MatchesRegex(pattern=email_pattern, name="contains_email")# Phone number detection (US format)phone_pattern = re.compile(r"\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}")contains_phone = MatchesRegex(pattern=phone_pattern, name="contains_phone")# Code block detection (markdown)code_block_pattern = re.compile(r"```[\s\S]*?```")contains_code = MatchesRegex(pattern=code_block_pattern, name="contains_code_block")# JSON object detectionjson_pattern = re.compile(r"\{[^{}]*\}")contains_json = MatchesRegex(pattern=json_pattern, name="contains_json")

Using String Patterns#

You can pass patterns as strings instead of compiled regex:

from phoenix.evals.metrics import MatchesRegex# String pattern (will be compiled automatically)date_evaluator = MatchesRegex(    pattern=r"\d{4}-\d{2}-\d{2}",    name="contains_date")eval_input = {"output": "The event is scheduled for 2024-03-15."}scores = date_evaluator.evaluate(eval_input)print(scores[0].score)  # 1.0

Using with Phoenix#

Evaluating Traces#

Run evaluations on traces collected in Phoenix and log results as annotations:

Evaluating Phoenix Traces

Running Experiments#

Use the MatchesRegex evaluator in Phoenix experiments:

Using Evaluators in Experiments

API Reference#

Python: MatchesRegex

Exact Match Evaluator - For exact string comparison

Overview#

When to Use#

Supported Levels#

Input Requirements#

Constructor Arguments#

Output Interpretation#

Usage Examples#

Common Pattern Examples#

Using String Patterns#

Using with Phoenix#

Evaluating Traces#

Running Experiments#

API Reference#

Related#