Skip to main content

agrag.eval.trajectory.expected_tools_metric

expected_tools_metric(names:Sequence[str], *, threshold:float = 0.5) -> ScoreMetric

Build the metric that the run called every expected tool.

Compares the trajectory's tool calls with the expected names as a superset, ignoring arguments: extra tools do not matter, a missing name fails. Scores 1.0 or 0.0.

Parameters:

  • names (Sequence[str]) – The tool names the run must include.
  • threshold (float) – The minimum score that counts as success.

Returns:

  • ScoreMetric – A ScoreMetric that scores tool presence.