Misattribution

Detect when LLMs misattribute facts in marketing text. Improve reliability and avoid misleading outputs.

Exclusive to enterprise customers. Contact us to activate this feature.

Objective: The Misattribution metric detects instances where quotes, information, or ideas are incorrectly attributed to sources not supported by the context. This metric is crucial for identifying situations where a quote or idea is erroneously attributed to someone other than the correct person, ensuring that the content is accurate before being posted to LinkedIn.

Required Columns in Dataset:

• Prompt: The original request or topic that led to the creation of the content.

• Context: The background information or source material used to generate the content.

• Response: The content generated by the model that is being evaluated for misattribution.

Score Range: 0 (low misattribution) to 1 (high misattribution)

Additional Information: Reasons for the score are provided along with the metric value to help understand and correct any misattributions.

Code Implementation

metrics = [
    {"name": "Misattribution", "config": {"model": "gpt-4o-mini", "provider":"azure", "key": "value"}, "column_name":"Response_Correctness_v2"},
    {"name": "Misattribution", "config": {"model": "gpt-4o-mini", "provider":"openai", "key":"value"}, "column_name":"Response_Correctness_v2"}
]

Example:

Prompt: Write a LinkedIn post discussing the importance of innovation in business, referencing Steve Jobs.

Context: Steve Jobs emphasized the importance of innovation in business, stating that "Innovation distinguishes between a leader and a follower."

Response: As Albert Einstein once said, "Innovation distinguishes between a leader and a follower." In today's fast-paced business world, staying ahead requires constant innovation...

Metric Score: Score: 0.9/1.0

Reasoning:

• Misattribution: The quote "Innovation distinguishes between a leader and a follower" is incorrectly attributed to Albert Einstein instead of Steve Jobs. This significant misattribution affects the credibility of the content.

• Correction Needed: The content needs to correctly attribute the quote to Steve Jobs to ensure accuracy and maintain trustworthiness.

Interpretation: The high score indicates a severe instance of misattribution, where a quote has been incorrectly attributed to a prominent figure. This error must be corrected to prevent the spread of misinformation and to maintain the integrity of the content on LinkedIn.