LogoLogo
Slack CommunityCatalyst Login
  • Welcome
  • RagaAI Catalyst
    • User Quickstart
    • Concepts
      • Configure Your API Keys
      • Supported LLMs
        • OpenAI
        • Gemini
        • Azure
        • AWS Bedrock
        • ANTHROPIC
      • Catalyst Access/Secret Keys
      • Enable Custom Gateway
      • Uploading Data
        • Create new project
        • RAG Datset
        • Chat Dataset
          • Prompt Format
        • Logging traces (LlamaIndex, Langchain)
        • Trace Masking Functions
        • Trace Level Metadata
        • Correlating Traces with External IDs
        • Add Dataset
      • Running RagaAI Evals
        • Executing Evaluations
        • Compare Datasets
      • Analysis
      • Embeddings
    • RagaAI Metric Library
      • RAG Metrics
        • Hallucination
        • Faithfulness
        • Response Correctness
        • Response Completeness
        • False Refusal
        • Context Relevancy
        • Context Precision
        • Context Recall
        • PII Detection
        • Toxicity
      • Chat Metrics
        • Agent Quality
        • Instruction Adherence
        • User Chat Quality
      • Text-to-SQL
        • SQL Response Correctness
        • SQL Prompt Ambiguity
        • SQL Context Ambiguity
        • SQL Context Sufficiency
        • SQL Prompt Injection
      • Text Summarization
        • Summary Consistency
        • Summary Relevance
        • Summary Fluency
        • Summary Coherence
        • SummaC
        • QAG Score
        • ROUGE
        • BLEU
        • METEOR
        • BERTScore
      • Information Extraction
        • MINEA
        • Subjective Question Correction
        • Precision@K
        • Chunk Relevance
        • Entity Co-occurrence
        • Fact Entropy
      • Code Generation
        • Functional Correctness
        • ChrF
        • Ruby
        • CodeBLEU
        • Robust Pass@k
        • Robust Drop@k
        • Pass-Ratio@n
      • Marketing Content Evaluation
        • Engagement Score
        • Misattribution
        • Readability
        • Topic Coverage
        • Fabrication
      • Learning Management System
        • Topic Coverage
        • Topic Redundancy
        • Question Redundancy
        • Answer Correctness
        • Source Citability
        • Difficulty Level
      • Additional Metrics
        • Guardrails
          • Anonymize
          • Deanonymize
          • Ban Competitors
          • Ban Substrings
          • Ban Topics
          • Code
          • Invisible Text
          • Language
          • Secret
          • Sentiment
          • Factual Consistency
          • Language Same
          • No Refusal
          • Reading Time
          • Sensitive
          • URL Reachability
          • JSON Verify
        • Vulnerability Scanner
          • Bullying
          • Deadnaming
          • SexualContent
          • Sexualisation
          • SlurUsage
          • Profanity
          • QuackMedicine
          • DAN 11
          • DAN 10
          • DAN 9
          • DAN 8
          • DAN 7
          • DAN 6_2
          • DAN 6_0
          • DUDE
          • STAN
          • DAN_JailBreak
          • AntiDAN
          • ChatGPT_Developer_Mode_v2
          • ChatGPT_Developer_Mode_RANTI
          • ChatGPT_Image_Markdown
          • Ablation_Dan_11_0
          • Anthropomorphisation
      • Guardrails
        • Competitor Check
        • Gibberish Check
        • PII
        • Regex Check
        • Response Evaluator
        • Toxicity
        • Unusual Prompt
        • Ban List
        • Detect Drug
        • Detect Redundancy
        • Detect Secrets
        • Financial Tone Check
        • Has Url
        • HTML Sanitisation
        • Live URL
        • Logic Check
        • Politeness Check
        • Profanity Check
        • Quote Price
        • Restrict Topics
        • SQL Predicates Guard
        • Valid CSV
        • Valid JSON
        • Valid Python
        • Valid Range
        • Valid SQL
        • Valid URL
        • Cosine Similarity
        • Honesty Detection
        • Toxicity Hate Speech
    • Prompt Playground
      • Concepts
      • Single-Prompt Playground
      • Multiple Prompt Playground
      • Run Evaluations
      • Using Prompt Slugs with Python SDK
      • Create with AI using Prompt Wizard
      • Prompt Diff View
    • Synthetic Data Generation
    • Gateway
      • Quickstart
    • Guardrails
      • Quickstart
      • Python SDK
    • RagaAI Whitepapers
      • RagaAI RLEF (RAG LLM Evaluation Framework)
    • Agentic Testing
      • Quickstart
      • Concepts
        • Tracing
          • Langgraph (Agentic Tracing)
          • RagaAI Catalyst Tracing Guide for Azure OpenAI Users
        • Dynamic Tracing
        • Application Workflow
      • Create New Dataset
      • Metrics
        • Hallucination
        • Toxicity
        • Honesty
        • Cosine Similarity
      • Compare Traces
      • Compare Experiments
      • Add metrics locally
    • Custom Metric
    • Auto Prompt Optimization
    • Human Feedback & Annotations
      • Thumbs Up/Down
      • Add Metric Corrections
      • Corrections as Few-Shot Examples
      • Tagging
    • On-Premise Deployment
      • Enterprise Deployment Guide for AWS
      • Enterprise Deployment Guide for Azure
      • Evaluation Deployment Guide
        • Evaluation Maintenance Guide
    • Fine Tuning (OpenAI)
    • Integration
    • SDK Release Notes
      • ragaai-catalyst 2.1.7
  • RagaAI Prism
    • Quickstart
    • Sandbox Guide
      • Object Detection
      • LLM Summarization
      • Semantic Segmentation
      • Tabular Data
      • Super Resolution
      • OCR
      • Image Classification
      • Event Detection
    • Test Inventory
      • Object Detection
        • Failure Mode Analysis
        • Model Comparison Test
        • Drift Detection
        • Outlier Detection
        • Data Leakage Test
        • Labelling Quality Test
        • Scenario Imbalance
        • Class Imbalance
        • Active Learning
        • Image Property Drift Detection
      • Large Language Model (LLM)
        • Failure Mode Analysis
      • Semantic Segmentation
        • Failure Mode Analysis
        • Labelling Quality Test
        • Active Learning
        • Drift Detection
        • Class Imbalance
        • Scenario Imbalance
        • Data Leakage Test
        • Outlier Detection
        • Label Drift
        • Semantic Similarity
        • Near Duplicates Detection
        • Cluster Imbalance Test
        • Image Property Drift Detection
        • Spatio-Temporal Drift Detection
        • Spatio-Temporal Failure Mode Analysis
      • Tabular Data
        • Failure Mode Analysis
      • Instance Segmentation
        • Failure Mode Analysis
        • Labelling Quality Test
        • Drift Detection
        • Class Imbalance
        • Scenario Imbalance
        • Label Drift
        • Data Leakage Test
        • Outlier Detection
        • Active Learning
        • Near Duplicates Detection
      • Super Resolution
        • Semantic Similarity
        • Active Learning
        • Near Duplicates Detection
        • Outlier Detection
      • OCR
        • Missing Value Test
        • Outlier Detection
      • Image Classification
        • Failure Mode Analysis
        • Labelling Quality Test
        • Class Imbalance
        • Drift Detection
        • Near Duplicates Test
        • Data Leakage Test
        • Outlier Detection
        • Active Learning
        • Image Property Drift Detection
      • Event Detection
        • Failure Mode Analysis
        • A/B Test
    • Metric Glossary
    • Upload custom model
    • Event Detection
      • Upload Model
      • Generate Inference
      • Run tests
    • On-Premise Deployment
      • Enterprise Deployment Guide for AWS
      • Enterprise Deployment Guide for Azure
  • Support
Powered by GitBook
On this page
  • System Requirements
  • Software Requirements
  • Sample Terraform Templates to Launch VM
  • Install Raga Catalyst on VM

Was this helpful?

  1. RagaAI Catalyst
  2. On-Premise Deployment

Evaluation Deployment Guide

PreviousEnterprise Deployment Guide for AzureNextEvaluation Maintenance Guide

Last updated 2 months ago

Was this helpful?

  1. VM Preparation

    • Launch a Virtual Machine (VM) using your organization's standard protocol

    • Ensure the VM meets all system requirements.

    • Ensure VM meets the Software Requirements & Network Validation.

    • If needed, consult cloud-specific documentation for VM setup or run our to launch VM in your cloud

    • Install nginx on the VM - and ensure that it is accessible from the developer machine. HTTP or HTTPS(if required) is already setup.

  2. Installation Execution

    • SSH into the instance

    • Run the provided bash script on your prepared VM

System Requirements

Architecture

ARM

AMD

CPU

8 vCPU

16 vCPU

Memory

32 GB

64 GB

Disk Space

128 GB

128 GB

  • Disk where /var is mounted should have 80GB as available disk space.

  • Linux user must have sudo access

Inbound Ports

  • 22 (SSH)

    • Required to access the terminal to run the script.

  • 80 (HTTP)

    • Required to access the APIs and UI.

Outbound Ports

  • 443 (HTTPS)

    • Required to pull images from docker hub.

    • Required to connect to hosted LLMs (e.g. OpenAPI, Anthropic)

  • 587 [Optional] (SMTP)

    • Only needed if email functionality is required.

  • 80 (HTTP)

    • Required only during initial setup. This is required to install few packages like (unizp, jq)

    • We should close port 80 after installing Raga Catalyst.

Software Requirements

  • Operating System: Ubuntu Noble 24.04 (LTS) or RHEL 9

Validating Installation & Network Accessibility

Verify Docker Installation & Networking

  • Run a Docker container:

    docker run --rm nginx sh -c "curl -o /dev/null -s -w '%{http_code}\n' https://www.google.com"
  • If it prints 200, the installation is successful, and the networking is compatible.

Verify Minikube Installation

  • Check the installation by executing:

    minikube version

Verify Nginx Installation

  • Check the Nginx status by executing:

    sudo systctl status nginx

    It should be running.

  • Execute the command:

    curl -o /dev/null -s -w '%{http_code}\n' localhost

    If should print 200

  • Access the default Nginx page from a browser outside the VM using its IP http://:80/

Sample Terraform Templates to Launch VM

Azure

Prerequisites

  • Ensure that your system supports Bash commands

  • Active Azure account with a valid subscription

  • Owner or Contributor role on the subscription

  • Registered namespaces:

    • Microsoft.Network

    • Microsoft.Compute

  • Ensure at least 16 vCPUs are available within your subscription and region to deploy the Standard Ds v3 Family instance type.

  • Azure CLI installed

Azure login using CLI

az login
  • Open the web link and input the code

  • Select the desired subscription

Execute the following command in your terminal:

curl -sL https://raga-catalyst-aws.s3.ap-south-1.amazonaws.com/create_vm.sh | bash -s -- 'azure' '<region>' '<vm_type>'

Replace placeholders:

  • <region>: Deployment region

  • <vm_type>: public | private

Output:

  • Get the IP and password of the Instance

SSH into the instance:

ssh -i ubuntu@<VM-IP>

Install Azure Monitor Agent (Optional)

Get the VM ID:

az vm show --name RagaCatalystVM --resource-group RagaCatalystRG --query "id" --output tsv

Example output:

/subscriptions/<tenant_id>/resourceGroups/RAGA-AMI/providers/Microsoft.Compute/virtualMachines/RagaCatalyst

This is the VM ID, which can be used in other Azure commands to reference the virtual machine

Install the Azure Monitor Agent on VM:

az vm extension set --name AzureMonitorLinuxAgent --publisher Microsoft.Azure.Monitor --enable-auto-upgrade true --ids <replace-id>
az network nsg rule delete \
    --resource-group RagaCatalystRG \
    --nsg-name RagaCatalyst-NSG \
    --name Allow-80-Outbound
AWS

Prerequisites

  • Ensure that your system supports Bash commands

  • AWS account:

    • You must have an AWS account with 8 vCPUs available and Ec2 FullAccess in that region in the region where the instance will be created. The AWS user must have programmatic access.

  • AWS user with programmatic access

  • AWS CLI installed

AWS login using CLI

Option A: Using aws configure

aws configure

Provide:

  • Access Key ID

  • Secret Access Key

  • Default Region (e.g., us-east-1)

  • Output Format (json, text, table)

Option B: Using AWS Keys Export

export AWS_ACCESS_KEY_ID=your-access-key-id
export AWS_SECRET_ACCESS_KEY=your-secret-access-key

Validate aws cli Login:

aws sts get-caller-identity

Execute the following command to Launch Raga Catalyst VM:

curl -sL https://raga-catalyst-aws.s3.ap-south-1.amazonaws.com/create_vm.sh | bash -s -- 'aws' '<region>' '<vm_type>'

Replace placeholders:

  • <region>: Deployment region

  • <vm_type>: public | private

Output:

  • Retrieve the PEM file raga-catalyst.pem from the current path. Copy the PEM file to another location as a precaution. If the PEM file is lost, you will not be able to SSH into the instances in the future

  • Save the security_group_id, as it will be required in the next steps.

  • Get the IP of the Instance

SSH into the instance:

ssh -i raga-catalyst.pem ubuntu@<VM-IP>
  • Get the security group ID from the step 2 command output and replace that ID in the following command security-group-id

  • replace your-region with your region

aws ec2 revoke-security-group-egress \
    --group-id <security-group-id> \
    --protocol tcp \
    --port 80 \
    --region <your-region> \
    --cidr 0.0.0.0/0
GCP

GCP Account Requirements

  • GCP Account , Project and Quotas

    • Ensure you have a Google Cloud account with an active project.

    • Ensure you have a Google Cloud project with a sufficient quota to launch an instance with the t2a-standard-8 machine type (8 vCPUs and 32 GiB Memory).

    • Confirm the availability of the required resources (such as VPCs, subnets, and public IP addresses) in your chosen region. Use This Command

      gcloud services enable compute.googleapis.com
    • Alternatively, enable it via the Google Cloud Console: Navigate to APIs & Services > Library, search for "Compute Engine API," and click Enable. Navigate to APIs & Services > Library, search for "Compute Engine API," and click Enable:

    • The Gcloud user or service account used must have the following permissions:

      • Compute Admin: To create and manage instances, networks, and firewall rules.

      • Storage Admin: If additional storage operations are required.

      • Service Account User: To assign the service account to the instance.

  • Programmatic Access

    • Set up programmatic access using either gcloud auth or service account credentials.

  • Google Cloud CLI Installed

    • Ensure the Google Cloud CLI (gcloud) is installed on your local system.

    • Installation guide: Install Google Cloud CLI.

GCP login using CLI

Option A: Using gcloud Auth

  • Authenticate with The Google Cloud SDK.

    gcloud auth application-default login
  • Authenticate using gcloud auth.

    gcloud auth login
  • Set the active project and region:

    gcloud config set project <PROJECT_ID>
    gcloud config set compute/region <REGION>
  • Export the Project ID in The Environment.

    export GOOGLE_PROJECT=$(gcloud config get-value project)
  • Verify your configuration:

    gcloud config list

Option B: Using a Service Account Key File

  • Obtain a service account key file (JSON format) with the necessary permissions.

    gcloud auth activate-service-account --key-file=<PATH_TO_KEY_FILE>
  • Set the active project and region:

    gcloud config set project <PROJECT_ID>
    gcloud config set compute/region <REGION>
  • Export the Project ID in The Environment.

    export GOOGLE_PROJECT=$(gcloud config get-value project)
  • Verify your configuration:

    gcloud config list

Execute the following command to Launch Raga Catalyst VM:

curl -sL https://raga-catalyst-aws.s3.ap-south-1.amazonaws.com/create_vm.sh | bash -s -- 'gcp' '<region>' '<vm_type>'

Replace placeholders:

  • <region>: Deployment region

  • <vm_type>: public | private

Output:

  • Retrieve the PEM file raga_ssh_private_key.pem from the current path. Copy the PEM file to another location as a precaution. If the PEM file is lost, you will not be able to SSH into the instances in the future

  • Get the IP of the Instance

SSH into the instance:

ssh -i raga_ssh_private_key.pem ubuntu@<VM-IP>

Install Raga Catalyst on VM

Install Raga Catalyst:

  • SSH into the instance and run the installation command.

  • Run the script received from inside the VM from user home directory.

Access Raga Catalyst

  • Wait 30 minutes after command execution

  • Portal: http://<vm_ip>

  • API: http://<vm_ip>/api

  • If you face any issues accessing Raga Catalyst, run the command cat /var/log/ragastartup.log and share the output with the Raga team for quick assistance

Docker 27.2.0+ ()

Minikube V1.33.0+ ()

Nginx ()

Install CLI command

Remove Port 80 Outbound Rule after on VM

Install CLI command

Remove Port 80 Outbound Rule after on VM

Refer to the section below for details.

Install Docker
Install Minikube
Install Nginx
terraform
terraform
System Requirements
Software Requirements
sample terraform templates
Verify Installation
installing Raga Catalyst
installing Raga Catalyst
Authentication Setup