mudler/LocalAI

Maintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/mudler/LocalAI.svg?style=card)](https://bestofgithub.dev/repo/mudler/LocalAI)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/mudler/LocalAI.svg)](https://bestofgithub.dev/repo/mudler/LocalAI)

Loading README…

Backend	What it does
vllm.cpp	From-scratch C++20 port of vLLM for text generation: paged KV cache, continuous batching, prefix caching, safetensors + GGUF loading, engine-enforced structured output, on CPU, CUDA, Metal and Vulkan. Also serves MiniMax-H3 joint video+audio generation
parakeet.cpp	C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription
moss-transcribe.cpp	C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass
moss-tts.cpp	C++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec
magpie-tts.cpp	C++/GGML port of NVIDIA's Magpie TTS Multilingual 357M: 22.05 kHz mono text-to-speech in 5 voices and 9+ languages, with the NanoCodec neural codec and tokenizer/G2P embedded in a single GGUF
ced.cpp	C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition
voice-detect.cpp	Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend
voxtral-tts.c	Mistral Voxtral-4B-TTS text-to-speech in pure C: 20 preset voices across 9 languages, 24 kHz WAV output, no dependencies beyond libc
vibevoice.cpp	Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization
rf-detr.cpp	Native RF-DETR object detection and instance segmentation
locate-anything.cpp	Open-vocabulary object detection and visual grounding (LocateAnything-3B)
depth-anything.cpp	Depth Anything 3 monocular metric depth + camera pose estimation
face-detect.cpp	Face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace), replacing the Python insightface backend
timestral.cpp	Mistral Timestral time-series forecasting: quantile forecasts with covariates, on the CPU
free-splatter.cpp	Pose-free 3D reconstruction (FreeSplatter): turns a handful of plain photos into 3D Gaussians, no camera poses or GPU required
trellis2.cpp	C++/GGML port of Microsoft TRELLIS.2: single-image to textured 3D mesh (GLB with PBR materials)
gem-x.cpp	Live human motion capture with SOMA and SMPL pose streams (CPU/Vulkan)
kimodo.cpp	C++/GGML text-to-motion on CPU and Vulkan, exported as animated skeleton GLB
privacy-filter.cpp	Standalone GGML PII/NER token-classification engine powering LocalAI's PII redaction tier
LocalVQE	Joint acoustic echo cancellation, noise suppression, and dereverberation
local-store	Local-first vector database for embeddings (shipped in-tree)

mudler/LocalAI

Guided tour

User and auth

Agents

Usage metrics per user

Fine-tuning and Quantization

WebRTC

Quickstart

macOS

Containers (Docker, podman, ...)

CPU only:

NVIDIA GPU:

AMD GPU (ROCm):

Intel GPU (oneAPI):

Vulkan GPU:

Loading models

Latest News

Features

Supported Backends & Acceleration

Backends built by us

Resources

Team

Citation

Sponsors

Individual sponsors

License

Acknowledgements

Contributors