> For the complete documentation index, see [llms.txt](https://docs.raga.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.raga.ai/ragaai-catalyst/ragaai-metric-library/code-generation/codebleu.md).

# CodeBLEU

Evaluate LLM-generated code with CodeBLEU. Capture logic, syntax, and structure beyond surface similarity.

**Objective:**

CodeBLEU is a comprehensive metric for evaluating code generation, integrating BLEU with code-specific aspects such as syntax and dataflow. It measures both linguistic and structural similarity, making it suitable for assessing code accuracy beyond surface-level token matching.

**Required Columns in Dataset:**

`Generated Code`, `Reference Code`

**Interpretation:**

* **High CodeBLEU Score:** Indicates strong alignment with the reference solution in terms of both syntax and logic.
* **Low CodeBLEU Score:** Reflects potential deviations in code structure or logic, which may impact functionality.

**Execution via UI:**

<figure><img src="https://1811327582-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYbIiNdp1QbG4avl7VShw%2Fuploads%2Fo4vcsgikR48WeVLbS2lx%2FScreenshot%202024-10-28%20at%204.49.03%E2%80%AFPM.png?alt=media&amp;token=0ce19529-c2de-4df6-837c-542fbbd903d1" alt=""><figcaption></figcaption></figure>

**Execution via SDK:**

```python
metrics=[
    {"name": "CodeBLEU", "schema_mapping": {"generated_code": "Generated Code", "reference_code": "Reference Code"}}
]

```