Factual package intelligence from PyPI

cute-tokenizer 1.0.2 package intelligence

Compact Unicode Token Encoding via Semantic-Anchored Byte-level BPE — a code-aware tokenizer

pip install cute-tokenizer

Current version
1.0.2
Python requirement
>=3.10
License
MIT
Distribution type
Includes compiled wheel
Release files
7 (6 wheels, 1 source)
Download size
26.2 MiB
Release history
7 releases with files
Median release cadence
1 days

cute-tokenizer dependencies

PyPI declares 17 unique dependency rules for this release. Environment markers are shown when supplied by the project.

  1. tokenizers <0.22,>=0.20
  2. transformers >=4.45
  3. regex >=2024.7.24
  4. pyahocorasick >=2.1
  5. tqdm >=4.66
  6. xxhash >=3.4
  7. orjson >=3.10

Compatibility and release files

cute-tokenizer publishes 6 wheels and 1 source archive for version 1.0.2. Wheel platform tags: macosx_10_12_x86_64, macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2, macosx_11_0_arm64, manylinux_2_17_aarch64.manylinux2014_aarch64, manylinux_2_17_x86_64.manylinux2014_x86_64, win_amd64.

Declared Python classifiers: 3.10, 3.11, 3.12.

Release activity and sources

PyPI lists 7 releases with files. The first dated release is ; 7 releases fall within the 365 days preceding the latest dated release. The current release files were uploaded on . The preceding dated release was .

6 historical releases are fully yanked in PyPI metadata.