Contextual Recall

Objective: The test measure the proportion of contexts which were had impact on the generated response

Required Parameters: Expected Response, Context

Interpretation: Higher score signifies major proportion of contexts supplied to the LLM were helpful in answering the prompt question

prompt = "Where and when was leonardo da vinci born?"
response = "Leonardo Da Vinci was born in Vinci, Italy in 1452."
expected_response = "In Vinci, in 1452"
context = [
    "Leonardo da Vinci (1452–1519) was an Italian polymath of the Renaissance period, renowned for his diverse talents in painting, sculpture, architecture, engineering, science, and invention.",
    "Born in Vinci, Italy, in 1452, Leonardo's artistic prowess is epitomized by iconic works such as the Mona Lisa and The Last Supper, which are globally recognized masterpieces.",
    "Apart from his artistic achievements, Leonardo made significant contributions to science, conducting pioneering studies in anatomy, engineering, mathematics, and physics. His anatomical drawings, ahead of their time, remain invaluable to medical science.",
    "Leonardo da Vinci's engineering designs were visionary, encompassing ideas for flying machines, military weaponry, and architectural innovations. While many of his inventions were not realized in his lifetime, they continue to inspire scientists and inventors today.",
    "Leonardo's interdisciplinary approach to knowledge and his relentless curiosity exemplify Renaissance humanism, emphasizing the potential of human intellect and creativity. His legacy continues to captivate people worldwide, leaving an enduring mark on Western culture and inspiring generations beyond his death in 1519.",
]

# contextual recall test
evaluator.add_test(
    test_names=["contextual_recall_test"],
    data={
        "prompt": prompt,
        "response": response,
        "expected_response": expected_response,
        "context": context_string,
    },
    arguments={"model": "gpt-4", "threshold": 0.6},
).run()

evaluator.print_results()