Ban Competitors

Scanner to detect competitors in a prompt. It uses a named-entity recognition model to extract organizations from the prompt and compares them to a list of competitors.

An effective competitor list should include:

• The official titles of all recognized competitors.

• Commonly used abbreviations or variations of these titles.

• Any subsidiaries or affiliated brands of the competitors.

Ensuring the thoroughness and precision of this list is crucial for the efficacy of the scanning system.

Considerations and Constraints

• Precision: The precision of competitor identification heavily relies on the capabilities of the Named Entity Recognition (NER) model and the inclusiveness of the competitor roster.

• Contextual Awareness: The scanner might not grasp the full context surrounding a competitor's name, potentially resulting in excessive redaction.

• Efficiency: The scanning procedure could introduce additional computational burden, particularly when dealing with extensive texts containing numerous entities.

Parameters:

data:

• prompt (str): Prompt to check for competitors.

arguments:

• competitors (Sequence[str]): List of competitors to detect.

• threshold (float, optional): Threshold to determine if a competitor is present in the prompt. Default is 0.5.

• redact (bool, optional): Whether to redact the competitor name. Default is True.

Interpretation:

The values matching the names of competitors from the competitors list should be replaced with [REDACTED] in the output.

Example:

evaluator.add_test(
    test_names=["ban_competitors_guardrail"],
    data={"prompt": """Given the personal profile of Johnathan Edward Doe, please summarise the following resume:

Profile:

- Full Name: Johnathan Edward Doe.
- Date of Birth: April 12, 1990.
- Address: 123 Elm Street, Springfield, IL, 62701.
- Email: [e-mail].
- Phone Number: [phone].
- Educational Background:
    - Springfield High School, Graduated in 2008;
    - Springfield University, B.Sc. Computer Science, Graduated in 2012.
- Employment:
    - ABC Tech Solutions, Software Engineer, 2012-2015;
    - OpenAI, Senior Software Engineer, 2015-2021;
    - XYZ Global Systems, Senior Software Developer, 2015-2021.""",
        },
    arguments = {"competitors" : ["XYZ Global Systems", "OpenAI"]}
).run()