← Back to the directory

EleutherAI/lm-evaluation-harness

View on GitHub

A framework for few-shot evaluation of language models.

EleutherAI/lm-evaluation-harness is #1,663 on the bestofgithub Elo ladder, in Emerald V, with a rating of 1,765 (−13 this week). It has 14,476 stars, 50 gained in the last week.

Emerald V

1,765 Elo

  • #1,663 on the ladder
  • +9 today · −13 this week
  • Peak 1,787
16 Elo to Emerald IV

Stars

14,476

  • +11 today
  • +50 this week
  • – this month

Repository

Forks
3.8k
Watchers
53
Language
Python
Last push

Achievements

bestofgithub badgeMaintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/EleutherAI/lm-evaluation-harness.svg?style=card)](https://bestofgithub.dev/repo/EleutherAI/lm-evaluation-harness)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/EleutherAI/lm-evaluation-harness.svg)](https://bestofgithub.dev/repo/EleutherAI/lm-evaluation-harness)

Install

pip
pipx install lm_eval

Similar repositories

codertimo/BERT-pytorch

Open on GitHubPython

Google AI 2018 BERT pytorch implementation

  • 1,337−1 Elo, change today
  • 6.5k stars
  • 1.3k forks
git
git clone https://github.com/codertimo/BERT-pytorch.git
  • bert
  • language-model
  • nlp
  • pytorch

OptimalScale/LMFlow

Open on GitHubPython

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

  • 1,362−1 Elo, change today
  • 8.4k stars
  • 820 forks
git
git clone https://github.com/OptimalScale/LMFlow.git
  • chatgpt
  • deep-learning
  • instruction-following
  • language-model

tatsu-lab/stanford_alpaca

Open on GitHubPython

Code and documentation to train Stanford's Alpaca models, and generate the data.

  • 1,610−6 Elo, change today
  • 30.2k stars
  • 3.9k forks
git
git clone https://github.com/tatsu-lab/stanford_alpaca.git
  • deep-learning
  • instruction-following
  • language-model

BlinkDL/RWKV-LM

Open on GitHubPython

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

  • 1,475−6 Elo, change today
  • 14.7k stars
  • 1k forks
git
git clone https://github.com/BlinkDL/RWKV-LM.git
  • attention-mechanism
  • chatgpt
  • deep-learning
  • gpt

README

Loading README…