flashinfer-ai/flashinfer
Open on GitHubCudaFlashInfer: Kernel Library for LLM Serving
- 1,668+4 Elo, change today
- 6.9k stars
- 1.7k forks
git clone https://github.com/flashinfer-ai/flashinfer.gitRepositories with the GitHub topic “cuda”: 30 repositories with 628k stars between them, mostly written in Python, C++ and Cuda. The highest-rated is debpalash/VoiceStudio, #10 on the bestofgithub Elo ladder.
FlashInfer: Kernel Library for LLM Serving
git clone https://github.com/flashinfer-ai/flashinfer.gitA Python framework for GPU-accelerated simulation, robotics, and machine learning.
git clone https://github.com/NVIDIA/warp.gitRun models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
git clone https://github.com/wici-ai/inferssd.gitVoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictation, transcription & audiobook creation in 646 languages.
git clone https://github.com/debpalash/VoiceStudio.gitAn interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.
git clone https://github.com/XuehaiPan/nvitop.gitSGLang is a high-performance serving framework for large language models and multimodal models.
git clone https://github.com/sgl-project/sglang.gitVoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictation, transcription & audiobook creation in 646 languages.
git clone https://github.com/debpalash/VoiceStudio.gitVoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing…
High-throughput LLM serving engine with PagedAttention and an OpenAI-compatible server.
The open-source AI voice studio. Clone, dictate, create.
SGLang is a high-performance serving framework for large language models and multimodal models.
World's fastest and most advanced password recovery utility
Bend 2: a fast language that blocks AI mistakes via proof. Install: curl -fsSL https://bend-lang.com/install.sh | sh
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports…
FlashInfer: Kernel Library for LLM Serving
Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency…
Open3D: A Modern Library for 3D Data Processing
CUDA Templates and Python DSLs for High-Performance Linear Algebra
CUDA on non-NVIDIA GPUs
Instant neural graphics primitives: lightning fast NeRF and more
Build and run Docker containers leveraging NVIDIA GPUs
Samples for CUDA Developers which demonstrates features in CUDA Toolkit
kaldi-asr/kaldi is the official location of the Kaldi project.
An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.
A Python framework for GPU-accelerated simulation, robotics, and machine learning.
NumPy & SciPy for GPU
NumPy aware dynamic Python compiler using LLVM
Solve puzzles. Learn CUDA.
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification…
cuDF - GPU DataFrame Library
Containers for machine learning
OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.
Go package for computer vision using OpenCV 4 and beyond. Includes support for DNN, CUDA, OpenCV Contrib, and OpenVINO.
An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.
Modular ZK(Zero Knowledge) backend accelerated by GPU
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.