Add Metric Corrections

Correct metric evaluations manually. Ensure fairness and accuracy in AI testing.

The Metric Correction feature allows you to refine automated, as well as custom evaluation metrics generated by LLMs on our platform. By providing human feedback—both in the form of corrected metric scores and accompanying explanations—you can improve the overall accuracy and reliability of the metrics. This feedback is later utilized as few-shot examples to enhance custom metric evaluations.

---

Overview

• Purpose: Enable users to adjust and improve automated metric scores (e.g., hallucination, context relevance) by providing human insights.

• Benefit: Over time, your corrections help the system learn and yield more accurate evaluations.

• Outcome: Enhanced metric reliability and improved custom metric performance through the integration of human-provided examples.

---

Getting Started

Accessing the Correction Interface

1. Locate the Data Point: In any dataset view, identify the metric you wish to correct.

2. Initiate Correction: Click the pen icon adjacent to the datapoint to open the correction popup.

Entering Your Correction

Within the correction popup, you can provide:

• Corrected Score: Input a new, numeric score that better reflects your evaluation.

• Feedback Explanation: Add a brief, textual explanation to describe your reasoning.

Note: Ensure that your corrected score is within the acceptable range for the metric, and that an explanation is provided when a correction is made.

---

Viewing and Managing Your Corrections

After submitting a correction:

• Visual Cues:

  • A pill icon labeled Corrected Score appears next to the original metric value.

  • Your textual explanation, titled "Feedback" is displayed just below the “Reasoning” row.

• Dataset Updates:

  • Columns for the corrected value and your feedback are added to the dataset.

  • Use the filtering options—Feedback Exists or Feedback Doesn’t Exist—to easily manage and review your inputs.

---

Frequently Asked Questions

• Can I track the feedback I provide? Yes. Corrections are visible directly within the dataset view, and you can filter records to see where feedback exists.

• Is it possible to change or remove my feedback later? The platform allows you to manage your feedback post-submission. Check the dataset view for options to update or remove your corrections if needed.

• Do I need to provide both a corrected score and an explanation? Both elements are encouraged to maximize the impact of your feedback. However, only the text explanation is mandatory on the platform.

---