sgl-project/sglang

Maintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/sgl-project/sglang.svg?style=card)](https://bestofgithub.dev/repo/sgl-project/sglang)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/sgl-project/sglang.svg)](https://bestofgithub.dev/repo/sgl-project/sglang)

Loading README…

Platform	Representative hardware
NVIDIA	B200/B300/GB200/GB300; H100/H200/H800/H20; A100; RTX 30/40/50 series, RTX 6000 Ada / PRO 6000; DGX Spark, Jetson Orin
AMD	Instinct MI300X, MI325X, MI350X, MI355X
Google TPU	v6e, v7; SGLang-TorchTPU / SGL-JAX
Intel (GPU / CPU)	Arc / Arc Pro B-Series GPUs, Xeon CPUs
Apple Silicon	Macs via Metal / MLX
Huawei Ascend	A2, A3, 950PR/DT NPUs
Moore Threads	MTT S5000 GPUs

Area	Projects	Purpose
Diffusion	SGLang Diffusion	Built into SGLang for image and video generation with diffusion models.
Audio	SGLang Omni	Audio model serving for text-to-speech (TTS) and automatic speech recognition (ASR).
Speculative Decoding	SpecForge	Train draft models for speculative decoding and deploy them with SGLang.
RL and Post-Training	Miles, slime, AReaL, Tunix, verl	Training frameworks that integrate SGLang for rollout generation.
Deployment and Orchestration	SMG, RBG, llm-d, Ray Serve, NVIDIA Dynamo	Deploy and scale SGLang inference services with routing, load balancing, and cluster orchestration.
Education	Mini-SGLang, zero-to-sglang, DeepLearning.AI course	Learn inference engine design and efficient text and image generation through code and hands-on courses.

SGLang: Fast inference for LLMs and multimodal models