Ban Substrings

Ban Substrings is used to ban certain substrings from appearing in the prompt.

Parameters:

data:

• prompt (str): Prompt to check for banned substrings.

arguments:

• substrings (Sequence[str]): List of substrings to ban.

• match_type (str): Type of match to perform. Can be either 'str' or 'word'. Default is 'str'.

• case_sensitive (bool, optional): Flag to indicate if the match should be case-sensitive. Default is False.

• redact (bool, optional): Flag to indicate if the banned substrings should be redacted. Default is False.

• contains_all (bool): Flag to indicate if need to match all substrings instead of any of them. Default is False, contains any.

Interpretation:

Values from the substrings list present in the prompt should not be included in the output. Passed if no substrings are present, Failed if substrings are present.

Example:

Passed Scenario -

evaluator.add_test(
    test_names=["ban_substrings_guardrail"],
    data={
        "prompt": """Given the personal profile of Johnathan Edward Doe, please summarise the following resume:

Profile:

- Full Name: Johnathan Edward Doe.
- Date of Birth: April 12, 1990.
- Address: 123 Elm Street, Springfield, IL, 62701.
- Email: [e-mail].
- Phone Number: [phone].
- Educational Background:
    - Springfield High School, Graduated in 2008;
    - Springfield University, B.Sc. Computer Science, Graduated in 2012.
- Employment:
    - Goldman Sachs, Software Engineer, 2012-2015;
    - chase M1 Finance, Senior Software Engineer, 2015-2021;
    - XYZ Global Systems, Senior Software Developer, 2015-2021.
""",
    },
    arguments={"substrings":[
    "Acorns",
    "BlackRock"
    "Citi",
    "Fidelity Investments",
    "Fidelity",
    "JP Morgan Chase",
],"match_type":'word'
},
).run()

##Result = No banned substrings found in the prompt.

Failed Scenario -

evaluator.add_test(
    test_names=["ban_substrings_guardrail"],
    data={
        "prompt": """Given the personal profile of Johnathan Edward Doe, please summarise the following resume:

Profile:

- Full Name: Johnathan Edward Doe.
- Date of Birth: April 12, 1990.
- Address: 123 Elm Street, Springfield, IL, 62701.
- Email: [e-mail].
- Phone Number: [phone].
- Educational Background:
    - Springfield High School, Graduated in 2008;
    - Springfield University, B.Sc. Computer Science, Graduated in 2012.
- Employment:
    - JP Morgan Chase, Software Engineer, 2012-2015;
    - chase M1 Finance, Senior Software Engineer, 2015-2021;
    - XYZ Global Systems, Senior Software Developer, 2015-2021.
""",
    },
    arguments={"substrings":[
    "Acorns",
    "XYZ Global Systems",
    "Citi",
    "Fidelity Investments",
    "Fidelity"
],"match_type":'word'
},
).run().print_results()

#Result = Banned substrings found in the prompt.