PII Detection RAG Metric

Check LLM outputs for Personally Identifiable Information (names, emails, phone numbers) to prevent sensitive data leaks and ensure privacy compliance.

Objective: This guardrail ensures that any given text does not leak any Personally Identifiable Information. Currently, PIIs supported are:- 'EMAIL_ADDRESS' and 'URL'. If PII is detected, it will fail with a reason that anonymises the detected PII entities.

Required Parameters: Response

Interpretation: Higher score indicates the model response has PII exposed.

Code Execution:

metrics=[
    {"name": "PII", "config": {"model": "gpt-4o-mini", "provider": "openai"}, "column_name": "your-column-identifier", "schema_mapping": schema_mapping}
]

Model and Provider parameters are not mandatory to run Guardrails.

The "schema_mapping" variable needs to be defined first and is a pre-requisite for evaluation runs. Learn how to set this variable here.

Example:

• Response: John Doe's MRI results were sent to [email protected].

• Metric Output: {‘score’: 1, ‘reason’: ‘John Doe’s MRI results were sent to EMAIL’}

Model and Provider parameter are not required to run Guardrails