Code Guardrail is used for scanning if the Markdown output of the language model includes code snippets in specific programming languages. It can be configured to allow or block specific programming languages.
Parameters:
data:
languages (Sequence[str]): The list of programming languages to allow or deny.
prompt (str): The prompt to scan for code snippets.
arguments:
is_blocked (bool, optional): Whether the languages are blocked or allowed. Default is True, blocked.
threshold (float, optional): The threshold for the risk score. Default is 0.5.
use_onnx (bool, optional): Whether to use ONNX for inference. Default is False.
Interpretation:
Failed if blocked language is detected, Passed if no blocked language is detected. If blocked language is detected, the code snippet is removed.
Example:
Passed Scenario -
evaluator.add_test(
test_names=["code_guardrail"],
data={
"prompt": """
Give me reasons why I should use Python over Java.
def calculate_risk_score(score: float, threshold: float) -> float:
if score > threshold:
return 1.0
risk_score = round(abs(score - threshold) / threshold, 1)
# Ensure risk score is between 0 and 1
return min(max(risk_score, 0), 1)
Java has better OOD and OOP concepts.
Python is more readable and has a simpler syntax.
""",
},
arguments={"languages": ["Python"], "is_blocked": True, "threshold": 0.5},
).run()
Result = "No markdown code snippets found in the text"
Failed Scenario -
evaluator.add_test(
test_names=["code_guardrail"],
data={
"prompt": """
Give me reasons why I should use Python over Java.
```
def calculate_risk_score(score: float, threshold: float) -> float:
if score > threshold:
return 1.0
risk_score = round(abs(score - threshold) / threshold, 1)
# Ensure risk score is between 0 and 1
return min(max(risk_score, 0), 1)
```
Java has better OOD and OOP concepts. Python is more readable and has a simpler syntax.
""",
},
arguments={"languages": ["Python"], "is_blocked": True, "threshold": 0.5},
).run()
Code Example:
evaluator.add_test(test_names=["code_guardrail"],data={"prompt": """Give me reasons why I should use Python over Java. ```def calculate_risk_score(score: float, threshold: float) -> float: if score > threshold: return 1.0 risk_score = round(abs(score - threshold) / threshold, 1) # Ensure risk score is between 0 and 1 return min(max(risk_score, 0), 1) ``` Java has better OOD and OOP concepts. Python is more readable and has a simpler syntax. """,},arguments={"languages": ["Python"], "is_blocked": True, "threshold": 0.5},).run()
