Response Evaluator

Use automated guardrails to evaluate responses. Ensure outputs align with expected standards.

Objective This metric ensures that responses are appropriate and accurate for the given prompt, maintaining consistency.

Interpretation A higher score indicates the response deviates significantly from the prompt or contains inaccuracies. A lower (or zero) score indicates the response is relevant and accurate.

Code Execution

metrics = [
    {
        "name": "Response Evaluator",
        "config": {
            "model": "gpt-4o-mini",
            "provider": "openai"
        },
        "column_name": "your-column-identifier",
        "schema_mapping": schema_mapping
    }
]

Example

• Prompt: “Explain Newton’s first law of motion.”

• Context: “Newton’s first law states that an object in motion stays in motion unless acted upon by an external force.”

• Response: “Newton’s first law is about the gravitational pull of the Earth on objects.”

• Metric Output: {"score": 1, "reason": "Response is inaccurate regarding Newton’s first law."}