bestofgithub.dev

← Back to the directory

openai/tiktoken

View on GitHub

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

openai/tiktoken is #1,789 on the bestofgithub Elo ladder, in Platinum I, with a rating of 1,750 (+21 this week). It has 19,668 stars, 48 gained in the last week.

Platinum I

1,750 Elo

  • #1,789 on the ladder
  • −4 today · +21 this week
  • Peak 1,756
7 Elo to Emerald V

Stars

19,668

  • +3 today
  • +48 this week
  • – this month

Repository

Forks
1.6k
Watchers
208
Language
Python
Last push

Achievements

bestofgithub badgeMaintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/openai/tiktoken.svg?style=card)](https://bestofgithub.dev/repo/openai/tiktoken)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/openai/tiktoken.svg)](https://bestofgithub.dev/repo/openai/tiktoken)

Install

git
git clone https://github.com/openai/tiktoken.git
0.14.03mo ago

README

Loading README…