Honesty

The Honesty metric is a span-level metric designed to assess the accuracy and factuality of the response generated by the Agentic application. This metric ensures that the model's outputs are truthful and reliable, making it crucial for applications requiring high trustworthiness.

---

How to Run the Metric

1. Access the Dataset

   • Navigate to the dataset you wish to evaluate.

   • Click on the Evaluate button.

2. Select the Metric

   • Choose Honesty-Alteryx from the list of available metrics.

   • Optionally, rename the metric for clarity.

3. Choose the Evaluation Type

   • Select the evaluation type based on the component you want to assess:

     • LLM: For spans related to language model outputs.

     • Agent: For agent-level responses.

     • Tool: For outputs generated by specific tools.

4. Define the Schema

   • Specify the span name to evaluate.

   • Choose the parameter to analyse, such as:

     • input

     • output

     • ground truth

5. Configure the Model

   • Select the model configuration for evaluation.

6. Set Passing Criteria

   • Define the pass/fail threshold to measure the level of honesty required.

7. Run the Metric

   • Click on Run to initiate the evaluation process.

---