Analysis

RagaAI Catalyst’s analysis dashboard helps interpret evaluation results with metrics, charts, and visualizations. Drill into test cases, spot failure patterns and gain insights to improve your LLM.

The Analysis section of the RagaAI Catalyst provides powerful data tables and visualizations that help you evaluate the performance of your dataset. Here, you can explore insights, metrics, and graphs to better understand the results of your testing and experimentation.

The Analysis tab is located within the following path: [Project > Dataset > Evaluation > Analysis]

This section allows you to:

• View insights related to various metrics and response columns.

• Toggle between different metrics for analysis.

• Add, customize, or remove graphs for deeper insight.

---

Insights Overview

Once you're in the Analysis tab, you’ll find a set of visual graphs and tables that summarize the results from your evaluation. Here's a step-by-step guide to navigating the insights:

1. Accessing the Insights

• In the Evaluation tab of your chosen dataset, click on Analysis.

• You will be presented with multiple graphs and insights that showcase data trends and key performance indicators.

2. Customizing Metrics

• By default, the dashboard shows all metrics applied to the dataset.

• You can toggle specific metrics using the dropdown menu at the top.

  • Tip: If you don’t want to view a certain metric’s analysis, simply uncheck it from the dropdown.

  

  metric selection

3. Summary Tables

• Table 1: Provides an overall summary of the dataset's response columns and metadata associated with it.

  • Positive Feedback: Calculated as the ratio of data points with positive feedback to the total feedback (positive + negative), shown as a percentage.

  • Note: i icon show the statistics of no. of positive and negative feedback datapoints

  
• Table 2: Displays a summary of all metrics that were applied to the dataset.

  • Shows the average score and the pass rate (percentage of data points that passed the metric threshold).

  • Note: The data in the table will automatically update based on any filters you apply.

  

  Metric Summary

---

Adding and Customizing Graphs

In addition to the default graphs, you can add new graphs to visualize specific metrics and metadata. Here’s how:

1. Adding a New Graph

• Click on the "Add Graph" button.

• Choose the type of graph you want to create (eg: metadata vs metric).

• Select your X-axis and Y-axis data points based on the metrics and metadata you wish to analyze.

• Click "Save", and your new graph will be generated.

2. Deleting or Editing Graphs

• If you want to remove a graph, click on the kebab menu on the top right of the graph.

• Select "Delete" to remove it.

• You can always add a graph back later by following the steps in the Adding a New Graph section.

---

By following these steps, you can easily access valuable insights and customize your visualizations to suit your analysis needs. The flexibility of the Analysis dashboard allows you to focus on the metrics that matter most, helping you derive actionable conclusions from your dataset.