Factual package intelligence from PyPI

minference 0.1.6.0 package intelligence

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

pip install minference

Current version
0.1.6.0
Python requirement
>=3.8.0
License
MIT License
Distribution type
No wheel classification available
Release files
1 (0 wheels, 1 source)
Download size
407.3 KiB
Release history
8 releases with files
Median release cadence
5 days

minference dependencies

PyPI declares 20 unique dependency rules for this release. Environment markers are shown when supplied by the project.

  1. transformers >=4.37.0
  2. torch
  3. triton
  4. einops

Compatibility and release files

minference publishes 0 wheels and 1 source archive for version 0.1.6.0. Wheel platform tags: No wheel platform tags.

No version-specific Python classifiers are declared.

Release activity and sources

PyPI lists 8 releases with files. The first dated release is ; 8 releases fall within the 365 days preceding the latest dated release. The current release files were uploaded on . The preceding dated release was .