> For the complete documentation index, see [llms.txt](https://docs.raga.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.raga.ai/ragaai-catalyst/agentic-testing/compare-experiments.md).
# Compare Experiments
The **Compare Experiments** feature in RagaAI Catalyst enables users to analyze and contrast up to 3 experiments, helping identify differences and patterns across experiments. This feature is ideal for evaluating the performance and impact of changes across different experimental setups.
***
**Benefits of Experiment Comparison**
* **Comprehensive Analysis**: Identify patterns and dependencies by comparing performance metrics and behaviors across experiments.
* **Cost Optimization**: Gain insights into cost breakdowns and opportunities for efficiency improvements.
* **Versioning Insights**: Understand how different code versions and configurations impact overall outcomes.
***
**Steps to Compare Experiments**
1. **Navigate to the Dataset Page**
* Inside the dataset view, locate the section displaying the selected dataset name.
2. **Select Compare Dataset**
* Click on the **Compare Dataset** button to initiate the experiment comparison process.
3. **Choose Experiments**
* Select up to 2 experiments you wish to compare.
4. **Start Comparison**
* Click the **Compare Experiments** button to generate the **Diff View**.
5. **Baseline Experiment**
* You can change the baseline experiment to analyze the comparison relative to a different experiment.
***
**Features of the Diff View**
1. **Code Diff**
* Displays a side-by-side comparison of the code versions or configurations used in the experiments.
* Highlights differences, additions, and deletions.
* Allows toggling between multiple code versions for in-depth comparison.
2. **Analysis Section**
* **Tool Usage Count Bar Chart**:
* Visualizes how often each tool is used across experiments.
* Reveals patterns and dependencies in tool utilization.
* **Time vs. Tool Calls Chart**:
* Shows how tool execution times vary between experiments.
* Helps identify bottlenecks or performance discrepancies.
* **Token Consumption Analysis**:
* Compares model interactions across experiments in terms of token usage.
* Enables users to evaluate efficiency and resource allocation.
* **Cost Analysis**:
* Breaks down cost incurred in each experiment.
* Helps guide optimization decisions by identifying high-cost components.
***
**Why Compare Experiments?**
* **Data-Driven Insights**: Make informed decisions about model configurations, tool selection, and resource allocation.
* **Performance Benchmarking**: Establish baselines and measure improvements or regressions.
* **Experiment Optimization**: Refine workflows and configurations based on comparative data.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.raga.ai/ragaai-catalyst/agentic-testing/compare-experiments.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.