Hallucination

Objective: This metric measures the extent of the model hallucinating i.e. model is making up a response based on its imagination which is far from being true to the correct response.

Required Parameters: Response, Context

Interpretation: A higher score indicates the model response was hallucinated.

Code Execution:

experiment_manager = Experiment(project_name="project_name",
                                experiment_name="experiment_name",
                                dataset_name="dataset_name")

# Hallucination Test
response =  experiment_manager.add_metrics(
    metrics=[
        {"name":"hallucination", "config": {"model": "gpt-4o"}},
        {"name":"hallucination", "config": {"model": "gpt-4"}},
        {"name":"hallucination", "config": {"model": "gpt-3.5-turbo"}}
    ]
)

Results:

Last updated