Cosine Similarity

Compare AI outputs with reference text using cosine similarity. Improve semantic consistency and accuracy.

Objective Measures the similarity between two pieces of text, ensuring consistency and relevance.

Interpretation A higher score typically means a higher degree of similarity (depending on implementation). A lower score indicates less similarity.

Implementation note: Some may flip this scoring scale (i.e., 0 for identical texts and higher for differences). Ensure your usage matches your interpretation.

Code Execution

metrics = [
    {
        "name": "Cosine Similarity",
        "config": {
            "model": "gpt-4o-mini",
            "provider": "openai"
        },
        "column_name": "your-column-identifier",
        "schema_mapping": schema_mapping
    }
]

Example

• Prompt: “Summarize the product description.”

• Context: “Product X is a portable charger with a 10000mAh battery.”

• Response: “Product X is a small device that charges phones and has a 10000mAh battery.”

• Metric Output: {"score": 0.85, "reason": "High similarity to the context."}