Response Grade Score

Objective: The Response Grade Score metric evaluates the complexity of a given response. This score is used to determine how difficult it is for a language model to interpret and respond to the response accurately.

Required Parameters: Response

Interpretation:

  • Higher Score: Indicates that the response is more complex and difficult to interpret. This may be due to factors such as ambiguous language, multiple layers of meaning, or advanced vocabulary.

  • Lower Score: Suggests that the response is simpler and easier to understand, likely containing clear and straightforward language.

Code Execution

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")

response = experiment_manager.add_metrics(
    metrics=[
        {"name":"response_grade_score", "config": {"model": "gpt-4o"}},
        {"name":"response_grade_score", "config": {"model": "gpt-4"}},
        {"name":"response_grade_score", "config": {"model": "gpt-3.5-turbo"}}
    ]
)

Last updated