Ranked Contextual Precision

Objective:

Required Parameters: Prompt, Expected Response, Context

Interpretation: A higher score signifies more major proportion of contexts supplied to the LLM helped answer the prompt question

Code Execution:

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")

response = experiment_manager.add_metrics(
    metrics=[
        {"name":"contextual_precision_ranked", "config": {"model": "gpt-4o"}},
        {"name":"contextual_precision_ranked", "config": {"model": "gpt-4"}},
        {"name":"contextual_precision_ranked", "config": {"model": "gpt-3.5-turbo"}}
    ]
)

Last updated