bestofgithub.dev

← Back to the directory

NVIDIA/TensorRT-LLM

View on GitHub

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

NVIDIA/TensorRT-LLM is #1,865 on the bestofgithub Elo ladder, in Platinum I, with a rating of 1,742 (−19 this week). It has 15,079 stars, 40 gained in the last week.

Platinum I

1,742 Elo

  • #1,865 on the ladder
  • −10 today · −19 this week
  • Peak 1,773
15 Elo to Emerald V

Stars

15,079

  • +4 today
  • +40 this week
  • – this month

Repository

Forks
2.9k
Watchers
119
Language
Python
Last push
Category
AI & Agents

Achievements

bestofgithub badgeMaintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/NVIDIA/TensorRT-LLM.svg?style=card)](https://bestofgithub.dev/repo/NVIDIA/TensorRT-LLM)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/NVIDIA/TensorRT-LLM.svg)](https://bestofgithub.dev/repo/NVIDIA/TensorRT-LLM)

Install

git
git clone https://github.com/NVIDIA/TensorRT-LLM.git
v1.2.16mo ago

Similar repositories

vllm-project/vllm

PickOpen on GitHubPython

High-throughput LLM serving engine with PagedAttention and an OpenAI-compatible server.

  • 2,222−2 Elo, change today
  • 96.9k stars
  • 24.8k forks
  • 604 watchers
pip
pip install vllm
v0.33.0rc313d ago
  • llm
  • serving
  • gpu

sgl-project/sglang

Open on GitHubPython

SGLang is a high-performance serving framework for large language models and multimodal models.

  • 2,080+4 Elo, change today
  • 38.9k stars
  • 10.5k forks
  • 184 watchers
git
git clone https://github.com/sgl-project/sglang.git
v0.5.2313d ago
  • attention
  • blackwell
  • cuda
  • deepseek

liguodongiot/llm-action

Open on GitHubHTML

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

  • 1,695−6 Elo, change today
  • 25.2k stars
  • 2.8k forks
  • 200 watchers
git
git clone https://github.com/liguodongiot/llm-action.git
  • llm
  • llm-inference
  • llm-serving
  • llm-training

flashinfer-ai/flashinfer

Open on GitHubCuda

FlashInfer: Kernel Library for LLM Serving

  • 1,668+4 Elo, change today
  • 6.9k stars
  • 1.7k forks
git
git clone https://github.com/flashinfer-ai/flashinfer.git
  • attention
  • cuda
  • distributed-inference
  • gpu

facebookresearch/fairseq

Open on GitHubPython

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

  • 1,632−8 Elo, change today
  • 32.2k stars
  • 6.6k forks
  • archived
git
git clone https://github.com/facebookresearch/fairseq.git
  • artificial-intelligence
  • python
  • pytorch

README

Loading README…