bestofgithub.dev

← Back to the directory

BlinkDL/RWKV-LM

View on GitHub

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

BlinkDL/RWKV-LM is #5,967 on the bestofgithub Elo ladder, in Silver III, with a rating of 1,475 (−34 this week). It has 14,774 stars, 2 gained in the last week.

Silver III

1,475 Elo

  • #5,967 on the ladder
  • −6 today · −34 this week
  • Peak 1,562
7 Elo to Silver II

Stars

14,774

  • +0 today
  • +2 this week
  • – this month

Repository

Forks
1k
Language
Python
Last push
Category
AI & Agents

Achievements

bestofgithub badgeMaintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/BlinkDL/RWKV-LM.svg?style=card)](https://bestofgithub.dev/repo/BlinkDL/RWKV-LM)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/BlinkDL/RWKV-LM.svg)](https://bestofgithub.dev/repo/BlinkDL/RWKV-LM)

Install

git
git clone https://github.com/BlinkDL/RWKV-LM.git

Similar repositories

EleutherAI/gpt-neo

Open on GitHubPython

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

  • 1,372+2 Elo, change today
  • 8.2k stars
  • 955 forks
  • archived
git
git clone https://github.com/EleutherAI/gpt-neo.git
  • gpt
  • gpt-2
  • gpt-3
  • language-model

BlinkDL/ChatRWKV

Open on GitHubPython

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

  • 1,382−1 Elo, change today
  • 9.4k stars
  • 685 forks
git
git clone https://github.com/BlinkDL/ChatRWKV.git
  • chatbot
  • chatgpt
  • language-model
  • pytorch

EleutherAI/gpt-neox

Open on GitHubPython

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

  • 1,396±0 Elo, change today
  • 7.4k stars
  • 1.1k forks
git
git clone https://github.com/EleutherAI/gpt-neox.git
  • deepspeed-library
  • gpt-3
  • language-model
  • transformers

OptimalScale/LMFlow

Open on GitHubPython

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

  • 1,362−1 Elo, change today
  • 8.4k stars
  • 820 forks
git
git clone https://github.com/OptimalScale/LMFlow.git
  • chatgpt
  • deep-learning
  • instruction-following
  • language-model

microsoft/LoRA

Open on GitHubPython

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

  • 1,550−7 Elo, change today
  • 13.9k stars
  • 936 forks
git
git clone https://github.com/microsoft/LoRA.git
  • adaptation
  • deberta
  • deep-learning
  • gpt-2

lucidrains/vit-pytorch

Open on GitHubPython

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

  • 1,646−5 Elo, change today
  • 25.5k stars
  • 3.4k forks
git
git clone https://github.com/lucidrains/vit-pytorch.git
  • artificial-intelligence
  • attention-mechanism
  • computer-vision
  • image-classification

README

Loading README…