bestofgithub.dev

← Back to the directory

xorbitsai/inference

View on GitHub

Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.

xorbitsai/inference is #5,869 on the bestofgithub Elo ladder, in Silver III, with a rating of 1,479 (−2 this week). It has 9,658 stars, 12 gained in the last week.

Silver III

1,479 Elo

  • #5,869 on the ladder
  • +8 today · −2 this week
  • Peak 1,492
3 Elo to Silver II

Stars

9,658

  • +3 today
  • +12 this week
  • – this month

Repository

Forks
901
Language
Python
Last push
Category
AI & Agents

Achievements

bestofgithub badgeMaintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/xorbitsai/inference.svg?style=card)](https://bestofgithub.dev/repo/xorbitsai/inference)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/xorbitsai/inference.svg)](https://bestofgithub.dev/repo/xorbitsai/inference)

Install

git
git clone https://github.com/xorbitsai/inference.git

Similar repositories

huggingface/transformers

PickOpen on GitHubPython

The reference library for pretrained text, vision and audio models: load, fine-tune and run thousands of checkpoints.

  • 2,186−3 Elo, change today
  • 169k stars
  • 35.2k forks
  • 1.2k watchers
pip
pip install transformers
v5.23.02d ago
  • ml
  • nlp
  • pytorch

jundot/omlx

Open on GitHubPython

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

  • 1,911+1 Elo, change today
  • 23.6k stars
  • 2.1k forks
  • 113 watchers
git
git clone https://github.com/jundot/omlx.git
v0.8.0rc13d ago
  • apple-silicon
  • inference-server
  • llm
  • macos

intel/ipex-llm

Open on GitHubPython

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

  • 1,398−4 Elo, change today
  • 8.8k stars
  • 1.4k forks
  • archived
git
git clone https://github.com/intel/ipex-llm.git
  • gpu
  • llm
  • pytorch
  • transformers

facebookresearch/fairseq

Open on GitHubPython

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

  • 1,632−8 Elo, change today
  • 32.2k stars
  • 6.6k forks
  • archived
git
git clone https://github.com/facebookresearch/fairseq.git
  • artificial-intelligence
  • python
  • pytorch

README

Loading README…