# Robust Drop\@k

**Objective:**

Robust Drop\@k measures a model’s sensitivity to prompt perturbations, capturing the decline in code accuracy as prompt variations are introduced. This metric helps understand a model’s susceptibility to changes, essential for tasks requiring adaptability and reliability.

**Required Columns in Dataset:**

`Original Prompt`, `Perturbed Prompts`, `Generated Code`

**Interpretation:**

* **Low Robust Drop\@k:** Indicates stable code generation, even under varied prompt conditions, suggesting high adaptability.
* **High Robust Drop\@k:** Reflects sensitivity to prompt changes, which may reduce effectiveness in dynamic applications.

**Execution via UI:**

<figure><img src="https://1811327582-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYbIiNdp1QbG4avl7VShw%2Fuploads%2FXVfIhHtYUpTadqoUjfBw%2FScreenshot%202024-10-28%20at%204.50.47%E2%80%AFPM.png?alt=media&#x26;token=2f549f40-15c3-4313-a001-369854c43029" alt=""><figcaption></figcaption></figure>

**Execution via SDK:**

```python
metrics=[
    {"name": "Robust Drop@k", "schema_mapping": {"original_prompt": "Original Prompt", "perturbed_prompts": "Perturbed Prompts", "generated_code": "Generated Code"}}
]

```