Summary Fluency

Objective:

Summary Fluency evaluates the linguistic quality of the summary, focusing on its grammatical correctness, smoothness of phrasing, and readability. It measures whether the summary is coherent in language, free of awkward wording or disjointed structure, and whether it can be easily understood by a reader.

Required Columns in Dataset:

LLM Summary

Interpretation:

• High values: Indicate that the summary is well-written, grammatically correct, and easily readable.

• Low values: Suggest poor grammar, awkward phrasing, or readability issues, making the summary difficult to understand.

Execution via UI:

Execution via SDK:

metrics=[
    {"name": "Summary Fluency", "config": {"model": "gpt-4o-mini", "provider": "openai"}, "column_name": "your-text", "schema_mapping": schema_mapping}
]