agrag.chunking.token
The token strategy: fixed-size windows of tokens, with optional overlap.
Classes:
- TokenChunker – Cuts the text into windows of
chunk_sizetokens.
agrag.chunking.tokenThe token strategy: fixed-size windows of tokens, with optional overlap.
Classes:
chunk_size tokens.