Skip to main content

agrag.ingestion.stats.chunking.ChunkingStats

Bases: BaseModel

Chunking-stage results.

Attributes:

  • chunks_by_strategy (dict[str, int]) – Chunk counts per chunker name, so chunks that a fallback made are counted under <strategy>:<fallback>.
  • documents_by_rule (dict[str, int]) – Document counts per rule, keyed "rule 0", "rule 1" and so on, and "fallback".
  • matches (list[ChunkingMatch]) – One entry per chunked document, capped at 1000.
  • matches_total (int) – Matches recorded before capping.
  • matches_truncated (bool) – Whether matches was cut to the cap.

Functions:

chunks_by_strategy​

chunks_by_strategy: dict[str, int] = Field(default_factory=dict)

documents_by_rule​

documents_by_rule: dict[str, int] = Field(default_factory=dict)

from_matches​

from_matches(matches:list[ChunkingMatch]) -> ChunkingStats

Summarize per-document matches.

Parameters:

  • matches (list[ChunkingMatch]) – One match per chunked document, in chunking order.

Returns:

  • ChunkingStats – The counters over all matches and the matches up to the cap.

matches​

matches: list[ChunkingMatch] = Field(default_factory=list)

matches_total​

matches_total: int = 0

matches_truncated​

matches_truncated: bool = False