Cover
Objective: The test is intended to check what proportion of the words provided was covered by the model in the response
Required Parameters:
Response: The sentence generated by the model.
Concept_set: A list of words in their root form, each tagged with its corresponding part of speech (e.g., "_V" for verb, "_N" for noun, "_A" for adjective).
Interpretation: A higher score indicates that the model's response contains all concepts as specified in the concept set.
Example with Higher Score:
Concept set: ["cat_N", "sleep_V", "windowsill_N"]
Response: "A cat is sleeping on the windowsill." (All concepts are used in model response.)
Example with Lower Score:
Concept set: ["sky_N", "sun_N", "windowsill_N"]
Response: "The sky is blue and the sun is shining." (windowsill is not covered in response.)
Note: Always use words in their root form in the concept set to focus on parts of speech application rather than word conjugation or inflection.
Last updated