Entity Co-occurrence

Test how accurately LLMs connect related entities. Spot gaps in recognition and strengthen relationship extraction.

Objective:

Entity Co-occurrence measures the frequency with which specific entities (e.g., names, places, events) appear together within a dataset. It is used to evaluate the relationship modeling of NER systems or entity extraction models. High entity co-occurrence implies that the model successfully identifies relevant pairings of entities that often appear together, important for downstream tasks like relation extraction or knowledge graph construction.

Required Columns in Dataset:

Prompt, Retrieved Entities, Expected Entities

Interpretation:

• High Entity Co-occurrence: Suggests that the model correctly identifies and links related entities, reflecting an understanding of their relationships in the dataset.

• Low Entity Co-occurrence: Indicates that the model fails to detect frequent co-occurring entities, potentially missing important connections.

Execution via UI:

Execution via SDK:

Entity Co-occurrence doesn't require an LLM for computation.

metrics=[
    {"name": "Entity Cooccurrence", "column_name": "your-text", "schema_mapping": schema_mapping}
]