mozilla-ai/llamafile

• Copper V

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

• 840+64 Elo, change today
• 1.1k stars
• 99 forks
• 13h ago

git clone https://github.com/wici-ai/inferssd.git

• apple-silicon
• cuda
• gguf
• inference

• Silver II

High-speed Large Language Model Serving for Local Deployment

• 1,484+1 Elo, change today
• 9.8k stars
• 619 forks
• 6mo ago

git clone https://github.com/Tiiny-AI/PowerInfer.git

• large-language-models
• llama
• llm
• llm-inference

• Emerald II

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

• 1,854−3 Elo, change today
• 10.5k stars
• 1.7k forks
• 62 watchers
• 4h ago

pipx install soup-cli

• cli
• consumer-gpu
• dpo
• fine-tuning

• Gold I

ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.

• 1,612+5 Elo, change today
• 7.8k stars
• 1.1k forks
• 8m ago

git clone https://github.com/Osmantic/ODS.git

• ai-agents
• amd
• comfyui
• docker

• Copper V

Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).

• 885+17 Elo, change today
• 1.2k stars
• 112 forks
• 14h ago

git clone https://github.com/synchopate/llama-modchip.git

• ai-agents
• apple-silicon
• avx512
• claude-code