llm-d/llm-d
State of the art inference performance with modern accelerators on Kubernetes
A star delta has been computed for 61,707 of 62,640 tracked repositories in this period, and only those can be ranked in it.
A repository missing here has not been measured far enough back yet. That does not mean it did not move.
Star counts measured between 00:29 and 08:09 (BKK).
State of the art inference performance with modern accelerators on Kubernetes
FlashInfer: Kernel Library for LLM Serving
High Performance Serving Runtime For Interactive Super Intelligence System
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Every device brings a slice. Together they run the whole model. Peer-to-peer LLM inference across browser tabs: a from-scratch WebGPU engine and a WebRTC runtime that split a 27B model over the devices in a room.
Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.
Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes
That's all 7 repositories in this period.