Skip to main content

agrag.loaders.corpus.types.DecodedText

DecodedText(text:str, encoding:str, had_bom:bool, content_hash:str, char_count:int, line_count:int) -> None

Output of the four-step decode pipeline.

Attributes:

  • text (str) – The decoded text, normalized as ReadOptions.normalization says.
  • encoding (str) – The encoding used to decode the bytes.
  • had_bom (bool) – Whether the source started with a byte-order mark.
  • content_hash (str) – The sha256 hash of the normalized text.
  • char_count (int) – The number of characters in text.
  • line_count (int) – The number of lines in text.

char_count​

char_count: int

content_hash​

content_hash: str

encoding​

encoding: str

had_bom​

had_bom: bool

line_count​

line_count: int

text​

text: str