FlashML-org/FreeToken

• Diamond II

SGLang is a high-performance serving framework for large language models and multimodal models.

• 2,080+4 Elo, change today
• 38.9k stars
• 10.5k forks
• 184 watchers
• 32m ago

git clone https://github.com/sgl-project/sglang.git

• attention
• blackwell
• cuda
• deepseek

• Legendary #74
• Hall of Fame #45

Run Llama, Gemma, Qwen and other open models locally behind a simple CLI and REST API.

• 2,277−5 Elo, change today
• 187k stars
• 19.1k forks
• 1k watchers
• 4h ago

curl -fsSL https://ollama.com/install.sh | sh

• llm
• local
• inference

• Emerald III

Nano vLLM

• 1,816−7 Elo, change today
• 16.4k stars
• 2.8k forks
• 86 watchers
• 6mo ago

git clone https://github.com/GeeeekExplorer/nano-vllm.git

• deep-learning
• inference
• llm
• nlp

• Diamond I

High-throughput LLM serving engine with PagedAttention and an OpenAI-compatible server.

• 2,222−2 Elo, change today
• 96.9k stars
• 24.8k forks
• 604 watchers
• 46m ago

pip install vllm

• llm
• serving
• gpu

• Gold V

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

• 1,520±0 Elo, change today
• 11.1k stars
• 1.8k forks
• 1d ago

git clone https://github.com/triton-inference-server/server.git

• cloud
• datacenter
• deep-learning
• edge

• Diamond I
• Hall of Fame #52

The reference library for pretrained text, vision and audio models: load, fine-tune and run thousands of checkpoints.

• 2,186−3 Elo, change today
• 169k stars
• 35.2k forks
• 1.2k watchers
• 1h ago

pip install transformers

• ml
• nlp
• pytorch