Cosine Similarity
Objective Measures the similarity between two pieces of text, ensuring consistency and relevance.
Interpretation A higher score typically means a higher degree of similarity (depending on implementation). A lower score indicates less similarity.
Implementation note: Some may flip this scoring scale (i.e., 0 for identical texts and higher for differences). Ensure your usage matches your interpretation.
Code Execution
metrics = [
{
"name": "Cosine Similarity",
"config": {
"model": "gpt-4o-mini",
"provider": "openai"
},
"column_name": "your-column-identifier",
"schema_mapping": schema_mapping
}
]
Example
Prompt: “Summarize the product description.”
Context: “Product X is a portable charger with a 10000mAh battery.”
Response: “Product X is a small device that charges phones and has a 10000mAh battery.”
Metric Output:
{"score": 0.85, "reason": "High similarity to the context."}
Last updated
Was this helpful?