# Robust Pass\@k

**Objective:**

Robust Pass\@k assesses model robustness by evaluating generated code’s ability to pass test cases across multiple perturbations of the input prompt. This metric provides insights into a model’s stability and robustness when faced with variations, enhancing its reliability for critical tasks.

**Required Columns in Dataset:**

`Original Prompt`, `Perturbed Prompt`, `Generated Code`

**Interpretation:**

* **High Robust Pass\@k:** Suggests that the model-generated code maintains functional accuracy across varied prompts, indicating robustness.
* **Low Robust Pass\@k:** Reveals potential instability in code generation, as output varies in functional quality across perturbations.

**Execution via UI:**

<figure><img src="https://1811327582-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYbIiNdp1QbG4avl7VShw%2Fuploads%2FaV7P18Be2LJyz211UF3m%2FScreenshot%202024-10-28%20at%204.50.06%E2%80%AFPM.png?alt=media&#x26;token=a01752e4-be01-4597-9db0-3b1718a987db" alt=""><figcaption></figcaption></figure>

**Execution via SDK:**

```python
metrics=[
    {"name": "Robust Pass@k", "schema_mapping": {"original_prompt": "Original Prompt", "perturbed_prompts": "Perturbed Prompts", "generated_code": "Generated Code"}}
]

```