Skip to main content

agrag.eval.extraction.relation_quality_metric

relation_quality_metric(*, symmetric_labels:frozenset[str] = frozenset(), threshold:float = 0.0) -> ScoreMetric

Build a metric for relation triple F1 on one test case.

A relation counts only when both endpoints align to gold entities and the triple is in gold. Use a new metric for each case.

Parameters:

  • symmetric_labels (frozenset[str]) – Relation labels with no direction. Their two endpoints are sorted before comparison.
  • threshold (float) – The minimum case score that counts as success.

Returns:

  • ScoreMetric – A metric that scores exact relation F1 for one case.