Response Evaluator

Objective This metric ensures that responses are appropriate and accurate for the given prompt, maintaining consistency.

Interpretation A higher score indicates the response deviates significantly from the prompt or contains inaccuracies. A lower (or zero) score indicates the response is relevant and accurate.

Code Execution

metrics = [
    {
        "name": "Response Evaluator",
        "config": {
            "model": "gpt-4o-mini",
            "provider": "openai"
        },
        "column_name": "your-column-identifier",
        "schema_mapping": schema_mapping
    }
]

Example

• Prompt: “Explain Newton’s first law of motion.”

• Context: “Newton’s first law states that an object in motion stays in motion unless acted upon by an external force.”

• Response: “Newton’s first law is about the gravitational pull of the Earth on objects.”

• Metric Output: {"score": 1, "reason": "Response is inaccurate regarding Newton’s first law."}