# Compare Experiments

The **Compare Experiments** feature in RagaAI Catalyst enables users to analyze and contrast up to 3 experiments, helping identify differences and patterns across experiments. This feature is ideal for evaluating the performance and impact of changes across different experimental setups.

***

**Benefits of Experiment Comparison**

* **Comprehensive Analysis**: Identify patterns and dependencies by comparing performance metrics and behaviors across experiments.
* **Cost Optimization**: Gain insights into cost breakdowns and opportunities for efficiency improvements.
* **Versioning Insights**: Understand how different code versions and configurations impact overall outcomes.

<figure><img src="/files/1jU7z1AcRSGNbPacnuEt" alt=""><figcaption></figcaption></figure>

***

**Steps to Compare Experiments**

1. **Navigate to the Dataset Page**
   * Inside the dataset view, locate the section displaying the selected dataset name.
2. **Select Compare Dataset**
   * Click on the **Compare Dataset** button to initiate the experiment comparison process.
3. **Choose Experiments**
   * Select up to 2 experiments you wish to compare.
4. **Start Comparison**
   * Click the **Compare Experiments** button to generate the **Diff View**.
5. **Baseline Experiment**
   * You can change the baseline experiment to analyze the comparison relative to a different experiment.

***

**Features of the Diff View**

1. **Code Diff**
   * Displays a side-by-side comparison of the code versions or configurations used in the experiments.
   * Highlights differences, additions, and deletions.
   * Allows toggling between multiple code versions for in-depth comparison.
2. **Analysis Section**
   * **Tool Usage Count Bar Chart**:
     * Visualizes how often each tool is used across experiments.
     * Reveals patterns and dependencies in tool utilization.
   * **Time vs. Tool Calls Chart**:
     * Shows how tool execution times vary between experiments.
     * Helps identify bottlenecks or performance discrepancies.
   * **Token Consumption Analysis**:
     * Compares model interactions across experiments in terms of token usage.
     * Enables users to evaluate efficiency and resource allocation.
   * **Cost Analysis**:
     * Breaks down cost incurred in each experiment.
     * Helps guide optimization decisions by identifying high-cost components.

***

**Why Compare Experiments?**

* **Data-Driven Insights**: Make informed decisions about model configurations, tool selection, and resource allocation.
* **Performance Benchmarking**: Establish baselines and measure improvements or regressions.
* **Experiment Optimization**: Refine workflows and configurations based on comparative data.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.raga.ai/ragaai-catalyst/agentic-testing/compare-experiments.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.