Response Completeness

Objective: The Completeness metric measures how well a response fulfills the requirements of the given prompt. It evaluates whether the response covers all necessary aspects as expected.

Required Parameters: Response, Expected Response

Interpretation:

• 1: The response is complete, meaning it thoroughly addresses all parts of the prompt and meets the criteria set by the expected response.

• 0: The response is incomplete, indicating that it fails to fully address the prompt or misses key elements outlined in the expected response.

Code Execution:

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")

response = experiment_manager.add_metrics(
    metrics=[
        {"name":"Response Completeness", "config": {"reason": True, "model": "gpt-4o-mini", "batch_size" : 5, "provider": "OpenAI"}}
    ]
)

Refer Executing tests page to learn about Metric Configurations

Example:

• Prompt: Describe the water cycle.

• Expected Response (Ground Truth): The water cycle consists of several stages: evaporation, condensation, precipitation, and collection. Water from oceans, lakes, and rivers evaporates due to the sun's heat, turning into water vapour. This vapour rises and cools, condensing into clouds. When the clouds become heavy, precipitation occurs in the form of rain, snow, sleet, or hail. The water then collects in bodies of water, and the cycle repeats.

• Response: The water cycle includes evaporation and precipitation. Water from oceans evaporates and later falls back to the earth as rain.

• Metric Output: {‘score’: 0.2, ‘reason’: The response is incorrect because it does not cover all the key stages and details of the water cycle as expected.’}