Prompt Format

How to represent conversation prompts and responses when preparing your dataset, ensuring Catalyst correctly interprets each user query and assistant reply during evaluation.

In addition to uploading in chat format, users can upload conversations in a simplified Prompt-Response format, where each interaction is recorded in a single row. Catalyst will automatically structure the data into a full chat history, allowing users to access all functionalities for chat analysis and processing.

The Prompt-Response dataset should follow the schema below, with Function, Timestamp, and Model Name columns as optional fields:

Chat ID	Chat Seq	Prompt	Response	Function	Timestamp	Model Name
1	1	Hello	Hi there!	N/A	15-10-2024 10:00	Model_XYZ
1	2	What is AI?	AI stands for Artificial Intelligence.	N/A	15-10-2024 10:01	Model_XYZ
2	1	Book a ticket	Your ticket has been booked.			Model_ABC

Columns:

1. Chat ID (Required): A unique identifier for each conversation, grouping all rows related to a single chat session.

2. Chat Seq (Required): A sequence number for each prompt-response pair within a chat session, maintaining the correct conversation flow.

3. Prompt : The message or query initiated by the user or customer.

4. Response : The assistant’s reply to the user’s prompt.

5. Function (Optional): The name of any function called in response to the prompt (e.g., check_balance, book_ticket). If no function is used, this field can be left empty or marked as "N/A".

6. Timestamp (Optional): The exact date and time of each interaction in the format DD-MM-YYYY HH:MM. This field can be left blank if precise timing is unnecessary for your analysis.

7. Model Name (Optional Metadata): The name or identifier of the AI model used to generate the response. This metadata allows tracking of different model responses and is optional.

84KB

Prompt Format CSV.csv

Sample CSV