Add Metric Corrections
Last updated
Was this helpful?
Last updated
Was this helpful?
The Metric Correction feature allows you to refine automated, as well as custom evaluation metrics generated by LLMs on our platform. By providing human feedback—both in the form of corrected metric scores and accompanying explanations—you can improve the overall accuracy and reliability of the metrics. This feedback is later utilized as few-shot examples to enhance custom metric evaluations.
Purpose: Enable users to adjust and improve automated metric scores (e.g., hallucination, context relevance) by providing human insights.
Benefit: Over time, your corrections help the system learn and yield more accurate evaluations.
Outcome: Enhanced metric reliability and improved custom metric performance through the integration of human-provided examples.
Locate the Data Point: In any dataset view, identify the metric you wish to correct.
Initiate Correction: Click the pen icon adjacent to the datapoint to open the correction popup.
Within the correction popup, you can provide:
Corrected Score: Input a new, numeric score that better reflects your evaluation.
Feedback Explanation: Add a brief, textual explanation to describe your reasoning.
Note: Ensure that your corrected score is within the acceptable range for the metric, and that an explanation is provided when a correction is made.
After submitting a correction:
Visual Cues:
A pill icon labeled Corrected Score appears next to the original metric value.
Your textual explanation, titled "Feedback" is displayed just below the “Reasoning” row.
Dataset Updates:
Columns for the corrected value and your feedback are added to the dataset.
Use the filtering options—Feedback Exists or Feedback Doesn’t Exist—to easily manage and review your inputs.
Can I track the feedback I provide? Yes. Corrections are visible directly within the dataset view, and you can filter records to see where feedback exists.
Is it possible to change or remove my feedback later? The platform allows you to manage your feedback post-submission. Check the dataset view for options to update or remove your corrections if needed.
Do I need to provide both a corrected score and an explanation? Both elements are encouraged to maximize the impact of your feedback. However, only the text explanation is mandatory on the platform.
