Contextual Recall

Objective: The test measure the proportion of contexts which were had impact on the generated response

Required Parameters: Expected Response, Context

Interpretation: Higher score signifies major proportion of contexts supplied to the LLM were helpful in answering the prompt question

Code Execution:

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")

response = experiment_manager.add_metrics(
    metrics=[
        {"name":"contextual_recall", "config": {"model": "gpt-4o"}},
        {"name":"contextual_recall", "config": {"model": "gpt-4"}},
        {"name":"contextual_recall", "config": {"model": "gpt-3.5-turbo"}}
    ]
)

Last updated