Factual package intelligence from PyPI

indic-tokenizer 0.2.2 package intelligence

BPE tokenizer for Indic scripts (Devanagari, Bengali, Tamil, Telugu, and more)

pip install indic-tokenizer

Current version
0.2.2
Python requirement
>=3.9
License
Not declared
Distribution type
Pure Python wheel
Release files
2 (1 wheels, 1 source)
Download size
79.0 KiB
Release history
5 releases with files
Median release cadence
0 days

indic-tokenizer dependencies

PyPI declares 9 unique dependency rules for this release. Environment markers are shown when supplied by the project.

  1. regex >=2023.0
  2. numpy >=1.24
  3. pandas >=1.5
  4. pyarrow >=10
  5. tqdm >=4.0
  6. datasets >=2.0
  7. huggingface_hub >=0.16
  8. sentencepiece >=0.1.99
  9. transformers >=4.12.2

Compatibility and release files

indic-tokenizer publishes 1 wheel and 1 source archive for version 0.2.2. Wheel platform tags: any.

No version-specific Python classifiers are declared.

Metadata completeness

PyPI does not currently declare: license. PyDeps marks these fields as unknown instead of guessing values.

Release activity and sources

PyPI lists 5 releases with files. The first dated release is ; 5 releases fall within the 365 days preceding the latest dated release. The current release files were uploaded on . The preceding dated release was .