Skip to main content

agrag.eval.adapter.ScoreMetric

ScoreMetric(name:str, scorer:Callable[[LLMTestCase], ScoreResult], threshold:float = 0.5) -> None

Bases: BaseMetric

A DeepEval metric backed by a plain scoring function.

Use it for scores that DeepEval does not compute, such as F1 from scikit-learn, so they report through the same evaluate() call. If the scorer raises, the error is stored in error and raised again.

Attributes:

  • name – The metric name shown in reports.
  • scorer – The function that turns a test case into a ScoreResult.
  • threshold – The minimum score that counts as success.

Functions:

  • a_measure – Run measure; scoring functions are synchronous.
  • measure – Run the scorer and record its score, reason and breakdown.

a_measure​

a_measure(test_case:LLMTestCase, *args:Any, **kwargs:Any) -> float

Run measure; scoring functions are synchronous.

measure​

measure(test_case:LLMTestCase, *args:Any, **kwargs:Any) -> float

Run the scorer and record its score, reason and breakdown.

name​

name = name

scorer​

scorer = scorer

threshold​

threshold = threshold