Prompt Grade Score

Objective: The Prompt Grade Score metric evaluates the complexity of a given prompt. This score is used to determine how difficult it is for a language model to interpret and respond to the prompt accurately.

Required Parameters: Prompt

Interpretation:

  • Higher Score: Indicates that the prompt is more complex and difficult to interpret. This may be due to factors such as ambiguous language, multiple layers of meaning, or advanced vocabulary.

  • Lower Score: Suggests that the prompt is simpler and easier to understand, likely containing clear and straightforward language.

Code Execution:

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")

response = experiment_manager.add_metrics(
    metrics=[
        {"name":"prompt_grade_score", "config": {"model": "gpt-4o"}},
        {"name":"prompt_grade_score", "config": {"model": "gpt-4"}},
        {"name":"prompt_grade_score", "config": {"model": "gpt-3.5-turbo"}}
    ]
)

Last updated