Slack CommunityCatalyst LoginPrism Login

User Quickstart

This guide will help you get started with RagaAI Catalyst using any sample dataset, straight from the RagaAI GUI or the Python environment.

This Quickstart section demonstrates steps to get started on the Catalyst UI. For users trying to run Catalyst from their SDK, equivalent commands for each step mentioned below can be found in this Google Colab project.

1.Sign Up and Authentication

1.1 Sign Up

1.2 Setup API Keys for LLM Access

To start running evaluations on RagaAI Catalyst, you need specify the API keys to your LLM models. Follow these steps to set your API keys:

  1. Navigate to Settings -> API Keys.

  2. Enter your API key against the relevant supported model names by clicking "Edit". You can also setup keys for your custom gateways under the 'custom parameters' section.

  3. Click "Save" to store the API Key(s).

2.Create New Project

Create a new project for testing your LLM Application by clicking the "Create New Project" button on the platform homepage

3. Upload Dataset

RagaAI Catalyst enables users to ingest data in two broad ways: real-time tracing and static uploads of data (CSV, Pandas, etc.). Here we'll upload a sample dataset via a CSV. For alternate methods, please refer to the Uploading Data section.

For further details on schema mapping and other CSV upload information, refer this page.

4. Run Evaluations on the Datasets

This can be implemented on the UI as follows. For further details on schema mapping and other evaluation related information, refer this page.

  • Check out the list of Supported metrics.

  1. View Results - Dataset, Analysis and Embedding View

You can go over the results for individual datapoints in the dataset view. [Project > Dataset > Dataset]

For each metric computed on a datapoint, users can refer to the "Reasoning" column inside the datapoint view to understand the logic behind its calculation:

The Analysis tab is located within the following path: [Project > Dataset > Analysis]

This section allows you to:

  • View insights related to various metrics and response columns.

  • Toggle between different metrics for analysis.

  • Add, customize, or remove graphs for deeper insight.

The Embedding tab is located within the following path: [Project > Dataset > Embedding]

This section allows you to:

  • Generate insights related to various metrics based on prompt embeddings

  • Identify failure modes for the LLM application.

Last updated