Skip to main content

agrag.eval.verifier.VerdictReport

Bases: BaseModel

Scores of predicted verdicts against gold verdicts.

Attributes:

  • labels (list[str]) – The class order of confusion_matrix.
  • per_class (dict[str, ClassScores]) – Scores for each class.
  • macro_f1 (float) – The mean F1 over the three classes.
  • confusion_matrix (list[list[int]]) – Counts with gold classes as rows and predicted classes as columns. A prediction of ERROR is in no column.
  • errors (int) – The number of ERROR predictions. Each one is a miss for the gold class of its item.

confusion_matrix​

confusion_matrix: list[list[int]]

errors​

errors: int

labels​

labels: list[str]

macro_f1​

macro_f1: float

per_class​

per_class: dict[str, ClassScores]