Outlier Detection

The Outlier Detection Test in RagaAI is crucial for identifying anomalies in low-resolution and high-resolution datasets, separately.

Execute Test:

The code snippet provided outlines the process of setting up and executing an Outlier Detection Test in RagaAI, focusing on detecting data points that deviate significantly from the majority of your dataset.

Step 1: Define the Outlier Detection Rules

Begin by establishing the criteria for detecting outliers in your dataset.

rules = OcrAnomalyRules()
rules.add(type="anomaly_detection", dist_metric="DistanceMetric", threshold=0.2)

ocr_test = ocr_anomaly_test_analysis(test_session=test_session,
                             dataset_name = "ocr_dataset",
                             test_name = "ocr_anomaly_detection",
                             model = "ocr_model",
                             type = "ocr",
                             output_type="anomaly_detection",
                             rules = rules)

                                   
test_session.add()

test_session.run()

• OCRAnomalyRules(): Initialises the rules for outlier detection for OCR Usecases.

  • rules.add(): Adds a rule for detecting anomalies:

    • type: The type of detection, "anomaly_detection" in this case.

    • dist_metric: The distance metric used for detection, "DistsanceMetric" here, which is effective for identifying outliers in a multidimensional space.

    • threshold: The threshold value for the DistanceScore metric, with 0.2 being the cut-off for identifying outliers.

• ocr_anomaly_test_analysis(): Configures the outlier detection test with the following parameters:

  • test_session: The session object tied to your RagaAI project.

  • test_name: A name for the test, "Outlier-detection-test" in this case.

  • dataset_name: The name of the dataset you are analysing, "ocr_dataset" here.

  • model: Specifies the OCR model used for inferences, "ocr_model" here.

  • type: Specify the usecase, "ocr" here.

  • output_type: For OCR use cases use output_type = "anomaly_detection".

  • rules: The previously defined rules for outlier detection.

test_session.add(): Registers the outlier detection test within the session.

test_session.run(): Starts the execution of all tests in the session, including the outlier detection test.

By completing these steps, you have initiated an Outlier Detection Test for OCR application on the RagaAI Testing Platform.

After the test, carefully review the identified outliers to decide how best to handle them — whether to remove, adjust, or further investigate these data points.

Analysing Test Results

Test Overview

• Pie Chart: A visual summary showing the proportion of data points that passed or failed the set distance metric threshold.

Distance Score Analysis

• Bar Graph: Visualise the average distance score for failed data points, with volume details per class.

Assessing and Visualising Data

• Datagrid View: Examine images sorted by their distance score in descending order.

Interpreting Results

• Failed: The test will identify bounding boxes that fall within the distance score threshold, classifying them as "failed".

• Passed: Bounding boxes that exceed the distance score threshold are classified as "passed".

By adhering to these steps, you can effectively utilise RagaAI to detect and analyse outliers in your OCR datasets.