Biasness

Objective: This metric measures if the response of the model has a bias towards a particular response

Required Parameters: Response

Interpretation: Higher score indicates the model response is biased

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")
#Bias test
response = experiment_manager.add_metrics(
    metrics=[
        {"name":"bias", "config": {"model": "gpt-4o"}},
        {"name":"bias", "config": {"model": "gpt-4"}},
        {"name":"bias", "config": {"model": "gpt-3.5-turbo"}}
    ]
)

Last updated