SYSTRAN/faster-whisper
Faster Whisper transcription with CTranslate2
A star delta has been computed for 62,678 of 63,382 tracked repositories in this period, and only those can be ranked in it.
A repository missing here has not been measured far enough back yet. That does not mean it did not move.
Star counts measured between 02:22 and 23:59 (BKK).
Faster Whisper transcription with CTranslate2
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
State-of-the-art LLM compression, built for production inference with vLLM
Project Maya - run GLM-5.3-Flash (321B MoE) and DeepSeek-V4.1-Flash-1017 (748B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images
Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.
PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率，量化感知训练的精度
A vector index built on TurboQuant, written in Rust with Python bindings
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
RDNA-native LLM inference engine in Rust.
Janas-LLM: large mixture-of-experts language models on ordinary computers, in C. Experts streamed from NVMe, with or without a GPU.
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
Fast inference engine for Transformer models
Accessible large language models via k-bit quantization for PyTorch.
ScribeCrate: open-source, self-hosted Whisper speech-to-text server powered by faster-whisper. OpenAI-compatible transcription and English translation APIs, speaker diarization, JSON/SRT/VTT output, SSE streaming, offline operation with cached models, and Docker deployment on CPU (amd64/arm64) or NVIDIA CUDA (amd64).
PROMETHEVS: run the full DeepSeek-V4.1-Flash-1017 (510 GB) on a gaming PC with 64 GB RAM and a 12 GB GPU. PROMETHEVS engine (compress-then-correct), OpenAI-compatible local server, report and logs.
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
A pytorch quantization backend for optimum
llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
Halo is an open-source framework built by White Circle for training large language and multimodal models
PyTorch native quantization for training and inference
Build, train, optimize, and run computer vision models locally, from raw images to live inference. Open source, optimized for Intel® XPU (CUDA also supported).
Neural Network Compression Framework for enhanced OpenVINO™ inference
An official lightweight library for the RaBitQ algorithm and its applications in vector search.
A safetensors extension to efficiently store sparse quantized tensors on disk
A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)
A unified inference runtime for VLA models.
Brevitas: neural network quantization in PyTorch
Showing 30 of 37 repositories