Spatio-Temporal Drift Detection

The Spatio Drift Detection Test is designed to detect and understand the localized changes in data distribution revealing how data evolves geographically over time.

Execute Test:

test_session = TestSession(
    project_name="Cropin",
    run_name=run_name,
    access_key=ACCESS_KEY,
    secret_key=SECRET_KEY,
    host=HOST
)

edge_case_detection = spatio_temporal_drift_test(test_session=test_session,
                                                 test_name="Drift-detection-test",
                                                 train_dataset_name="train_dataset_spatio_v2",
                                                 field_dataset_name="test_dataset_spatio-v3",
                                                 train_embed_col_name="ImageEmbedding",
                                                 field_embed_col_name="ImageEmbedding",
                                                 model="imageModel",
                                                 level="image",
                                                 primary_metadata="place",
                                                 primary_metadata_type="categorical",
                                                 type="spatio-temporal",
                                                 rules=rules)
test_session.add(edge_case_detection)
test_session.run()

Initialize Drift Detection Rules:

• Use the DriftDetectionRules() function to set up the framework for anomaly detection.

• Add rules using rules.add() with parameters defining the metric for drift detection and the threshold for identifying significant drift.

Configure Test Settings:

• Clearly define both the training (historical data) and field (current data) datasets to analyze shifts over time and space.

• Include metadata parameters like primary_metadata to ensure geographical variables are accounted for, and set primary_metadata_type to "categorical" for accurate categorization.

Execute Test:

• Implement the spatio_temporal_drift_test() with necessary parameters:

  • test_session: The session managing tests.

  • train_dataset_name and field_dataset_name: Datasets to compare.

  • rules: Configured rules for detection.

  • test_name: Name of the test for tracking.

Add Test to Session:

• Register the configured test with the session using test_session.add().

Run Test:

• Start test execution across the session with test_session.run().

By following these steps, you have successfully set up and executed an Spatio-Temporal Drift Test on the RagaAI Testing Platform.

Post-execution, review the results to identify and remove or handle duplicates as necessary.

Analyzing Test Results:

• In Distribution Data Points: Identified as "in distribution" when they fall within the predefined threshold, indicating alignment with the spatio-temporal data.

• Out of Distribution Data Points: Labelled as "out of distribution" if they surpass the threshold, indicating potential drift that requires a thorough examination of the spatio-temporal data.

Interpreting Test Results:

• Embedding View: Utilize the Interactive Embedding View to visualize and understand the drift between spatio-temporal datasets interactively.

• Data Grid View: Helps visualise annotations with spatio-temporal images sorted by mistake scores.

• Image View: Delve into detailed analyses for each image.

Following these guidelines, you can effectively utilise Spatio Temporal Drift Detection in RagaAI to maintain the integrity and relevance of your models over time.