Gibberish Check

Detect nonsensical outputs from LLMs. Apply guardrails to ensure clarity and coherence.

Objective This metric identifies and filters out nonsensical or incoherent responses, ensuring high-quality outputs.

Interpretation A higher score indicates that the response contains gibberish or incoherence. A lower (or zero) score indicates the response is coherent.

Code Execution

metrics = [
    {
        "name": "Gibberish Check",
        "config": {
            "model": "gpt-4o-mini",
            "provider": "openai"
        },
        "column_name": "your-column-identifier",
        "schema_mapping": schema_mapping
    }
]

Example

• Prompt: “Explain how photosynthesis works.”

• Context: “Photosynthesis is the process by which plants convert sunlight into chemical energy.”

• Response: “Blarg foo sunlight munch munch!”

• Metric Output: {"score": 1, "reason": "Response is incoherent or nonsensical."}