Language Same

This scanner checks if the input and output are written in the same language.

Required Parameters: Prompt, Response

Interpretation: Result pass denotes that the prompt and response are in same language and vice-versa.

Code Example:

# LanguageSame
prompt = """Hello, how are you today? I hope you're doing well"""
neg_response = """Bonjour, comment vas-tu aujourd'hui ? J'espère que tu vas bien"""
pos_response = """I am doing good, how are you?"""

evaluator.add_test(
    test_names=["language_same_guardrail"],
    data={
        "prompt": prompt,
        "response": pos_response,
    },
    arguments={"model": "gpt-3.5-turbo-1106", "threshold": 1.0},
).add_test(
    test_names=["language_same_guardrail"],
    data={
        "prompt": prompt,
        "response": neg_response,
    },
    arguments={"model": "gpt-3.5-turbo-1106", "threshold": 1.0},
).run()

evaluator.print_results()