Plan Adaptability

Evaluate how well your agentic pipeline responds to any unforeseen events like errors, new information, etc.

Objective

This metric evaluates the ability of an AI system to effectively adjust its plans in response to changing circumstances, unexpected outcomes, or new information during task execution. It measures how well the AI maintains progress toward its goals while dynamically adapting to new challenges.

Interpretation

A score closer to 1.0 indicates that the AI consistently makes effective and timely adaptations to its plans, leading to successful task outcomes even in the face of unexpected events. A score closer to 0.0 suggests that the AI either fails to adapt or makes ineffective adjustments that hinder task completion.

Syntax

# Plan Adaptability
response = experiment_manager.execute(
    metrics=[
        {"name": "plan_adaptability", "config": {"model": "gpt-4o-mini"}}
    ]
)