A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Runs large mixture-of-experts models split across CPU and GPU, on top of its own optimized kernels.
- Language
- Python
- Licence
- Apache-2.0
- Stars
- 20k
- Latest
- v0.7.2
- Last push
- 2026-10-16