Skip to main content

agrag.eval.trajectory.retry_budget_metric

retry_budget_metric(max_attempts:int, *, threshold:float = 0.5) -> ScoreMetric

Build the metric that retries stay within budget.

Counts researcher task spans starting after the first verifier task span ended. A call the limiter blocks leaves no span, so only executed delegations count. Scores 1.0 or 0.0.

Parameters:

  • max_attempts (int) – How many researcher retries after verification pass.
  • threshold (float) – The minimum score that counts as success.

Returns:

  • ScoreMetric – A ScoreMetric that scores the retry budget.