Create Evaluator

Use createEvaluator when your evaluation logic is plain TypeScript and you want a reusable evaluator object with consistent metadata and telemetry.

Example#

import { createEvaluator } from "@arizeai/phoenix-evals";const exactMatch = createEvaluator(  ({ output, expected }) => ({    score: output === expected ? 1 : 0,    label: output === expected ? "match" : "mismatch",  }),  {    name: "exact-match",    kind: "CODE",  });const result = await exactMatch.evaluate({  output: "Paris",  expected: "Paris",});

What You Get#

an evaluator name
evaluator kind such as CODE or LLM
optimization direction metadata
optional OpenTelemetry spans around execution

When To Use Code Evaluators#

Code evaluators are the right fit when the scoring logic should stay deterministic, cheap, and fully under your control:

regex or exact-match checks
JSON structure validation
latency and cost thresholds
post-processing checks on existing model output

import { createEvaluator } from "@arizeai/phoenix-evals";const lengthCheck = createEvaluator(  ({ output }) => {    const score = typeof output === "string" && output.length < 280 ? 1 : 0;    return {      score,      label: score ? "fits-limit" : "too-long",    };  },  {    name: "response-length",    kind: "CODE",  });

asEvaluatorFn
toEvaluationResult

Create Evaluator

Relevant Source Files

Example#

What You Get#

When To Use Code Evaluators#

Source Map

Relevant Source Files

Example#

What You Get#

When To Use Code Evaluators#

Related Helpers#

Source Map