MINEA

Apply MINEA (Multiple Infused Needle Extraction Accuracy) to test subjective question correction. Identify biases and improve fairness in LLM responses.

Objective:

MINEA assesses information extraction models by measuring the accuracy of identifying rare but critical information (the "needles") from large datasets (the "haystack"). It uses a combination of rarity detection and contextual extraction to evaluate how well a model pulls specific, infrequent data points. A high MINEA score implies the model successfully extracts crucial, rare information without being overwhelmed by more common or irrelevant data.

Required Columns in Dataset:

Labeled Text, Source Document

Interpretation:

• High MINEA: Indicates that the model excels at identifying rare and crucial pieces of information, even when they are hidden within a large volume of irrelevant or common data.

• Low MINEA: Suggests the model struggles to detect infrequent yet important details, leading to potential information gaps in extraction.

Execution via UI:

Execution via SDK:

metrics=[
    {"name": "MINEA", "config": {"model": "gpt-4o-mini", "provider": "openai"}, "column_name": "your-text", "schema_mapping": schema_mapping}
]