Image Property Drift Detection

The Image Property Drift Detection Test is designed to monitor the shifts in individual image properties over time.

Execute Test:

rules = IPDRules()
rules.add(metric="image-property-suite")

edge_case_detection = image_property_drift(test_session=test_session,
                                           reference_dataset_name=reference_dataset_name,
                                           eval_dataset_name=eval_dataset_name,
                                           rules=rules,
                                           test_name="Image-Property-Drift",
                                           type="image-property-drift",
                                           output_type="image-data"
                                           )
test_session.add(edge_case_detection)
test_session.run()

Initialize Image Property Drift Rules:

• Use the IPDRules() function to initialise the rules for the test.

• Add Rules:

  • Use the rules.add() function to add the "image-property-suite" metric, designed to assess various image properties such as color distribution, brightness, and sharpness across datasets.

Configure Test Settings:

• Set up the image property drift test by specifying the reference and evaluation datasets to compare image properties over time.

• Define the type of test as "image-property-drift" and specify the output_type to indicate how results should be visualised or returned.

Execute Test:

• Use the image_property_drift() function to execute the test with the specified parameters:

  • test_session: The session object managing tests.

  • reference_dataset_name: Name of the dataset used as a reference.

  • eval_dataset_name: Name of the dataset used for evaluation.

  • rules: Predefined rules for the test.

  • test_name: Name of the test run.

Add Test to Session:

• Use the test_session.add() function to register the test within the session.

Run Test:

• Use the test_session.run() function to execute all tests added to the session, including the Image Property Drift Test.

By following these steps, you have successfully set up and execute an Image Property Drift Test on the RagaAI Testing Platform.

Post-execution, review the results to identify and remove or handle duplicates as necessary.

Analyzing Test Results:

The Image Property Drift Test results provide insights into the drifts in image properties between a reference dataset and an evaluation dataset. These insights are presented through visualizations and data grids aiding in the identification of changes in image property distribution over time.

Bar Chart Comparison:

• The bar chart compares the distribution of image properties between the reference dataset and the evaluation dataset.

• Analyze image property distributions to understand trends in the drift of image properties across datasets.

Data Grid View:

• Two data grid views are presented side by side, showcasing the reference dataset and evaluation dataset.

• Each grid shows annotated images sorted according to their label drift scores.

Image View:

• The Image View enables users to conduct in-depth analyses of the drift in image properties for individual images.

By following these steps, the Image Property Drift Test acts as a preventive tool, ensuring that models remain robust and accurate by adapting to changes in key image properties, thereby reducing potential biases and inaccuracies.