Logic Check

Identify logical flaws or contradictions in model responses. Improve reliability with automated checks.

Objective Ensures that the text follows logical consistency, preventing contradictory or nonsensical statements.

Interpretation A higher score indicates a logical inconsistency or contradiction in the response. A lower (or zero) score indicates the response is logically consistent.

Code Execution

metrics = [
    {
        "name": "Logic Check",
        "config": {
            "model": "gpt-4o-mini",
            "provider": "openai"
        },
        "column_name": "your-column-identifier",
        "schema_mapping": schema_mapping
    }
]

Example

• Prompt: “Explain how a car can be both moving and stationary at the same time.”

• Context: “We expect logically consistent or well-reasoned explanations.”

• Response: “The car is moving at 60 mph but also not moving at all.”

• Metric Output: {"score": 1, "reason": "Logical contradiction detected (car cannot be both moving and stationary simultaneously without context)."}