Logic Check
Objective Ensures that the text follows logical consistency, preventing contradictory or nonsensical statements.
Interpretation A higher score indicates a logical inconsistency or contradiction in the response. A lower (or zero) score indicates the response is logically consistent.
Code Execution
metrics = [
{
"name": "Logic Check",
"config": {
"model": "gpt-4o-mini",
"provider": "openai"
},
"column_name": "your-column-identifier",
"schema_mapping": schema_mapping
}
]
Example
Prompt: “Explain how a car can be both moving and stationary at the same time.”
Context: “We expect logically consistent or well-reasoned explanations.”
Response: “The car is moving at 60 mph but also not moving at all.”
Metric Output:
{"score": 1, "reason": "Logical contradiction detected (car cannot be both moving and stationary simultaneously without context)."}
Last updated
Was this helpful?