Evaluator Message Formats#
Phoenix evaluators now support flexible prompt formats in both Python and TypeScript, giving you full control over how you structure prompts for LLM-based evaluations.
Supported Formats#
String Templates - Simple templates with variable placeholders:
Message Lists - OpenAI-style arrays with role and content fields for multi-turn prompts:
Template Variable Syntax#
- Python: Supports both f-string (
{variable}) and mustache ({{variable}}) syntax with auto-detection - TypeScript: Uses mustache syntax (
{{variable}})
Provider Compatibility#
Adapters handle provider-specific message transformations automatically:
| Provider | Transformation |
|---|---|
| OpenAI | System role converted to developer role for reasoning models |
| Anthropic | System messages extracted to system parameter |
| Google GenAI | System messages passed via system_instruction |
| LiteLLM | Messages passed in OpenAI format (LiteLLM handles conversion) |
| LangChain | Converted to LangChain message objects |
More Information:#
Eval Prompt Templates

