Completeness

Objective:The Completeness metric measures how well a response fulfills the requirements of the given prompt. It evaluates whether the response covers all necessary aspects as expected.

Required Parameters: Prompt, Response, Expected Response

Interpretation:

  • 1: The response is complete, meaning it thoroughly addresses all parts of the prompt and meets the criteria set by the expected response.

  • 0: The response is incomplete, indicating that it fails to fully address the prompt or misses key elements outlined in the expected response.

Code Execution:

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")

response = experiment_manager.add_metrics(
    metrics=[
        {"name":"completeness_gt", "config": {"model": "gpt-4o"}},
        {"name":"completeness_gt", "config": {"model": "gpt-4"}},
        {"name":"completeness_gt", "config": {"model": "gpt-3.5-turbo"}}
    ]
)

Last updated