agrag.agents.middleware.RoundRobinModelMiddleware

RoundRobinModelMiddleware(models:list[Any]) -> None

Bases: AgentMiddleware

Rotate across the configured chat models, one model per call.

Overrides the request's model on every model call so requests are distributed across all configured clients in order.

Functions:

• awrap_model_call – Run the call against the next model in rotation.
• wrap_model_call – Run the call against the next model in rotation.

Parameters:

• models (list[Any]) – Chat models to rotate across, in configuration order. Must be non-empty.

Raises:

• ValueError – models is empty.

awrap_model_call

awrap_model_call(request:ModelRequest, handler:Callable[[ModelRequest], Awaitable[ModelResponse]]) -> Any

Run the call against the next model in rotation.

wrap_model_call

wrap_model_call(request:ModelRequest, handler:Callable[[ModelRequest], ModelResponse]) -> Any

Run the call against the next model in rotation.