rasbt/LLMs-from-scratch

Maintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/rasbt/LLMs-from-scratch.svg?style=card)](https://bestofgithub.dev/repo/rasbt/LLMs-from-scratch)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/rasbt/LLMs-from-scratch.svg)](https://bestofgithub.dev/repo/rasbt/LLMs-from-scratch)

Loading README…

Chapter Title	Main Code (for Quick Access)	All Code + Supplementary
Setup recommendations How to best read this book	-	-
Ch 1: Understanding Large Language Models	No code	-
Ch 2: Working with Text Data	- ch02.ipynb - dataloader.ipynb (summary) - exercise-solutions.ipynb	./ch02
Ch 3: Coding Attention Mechanisms	- ch03.ipynb - multihead-attention.ipynb (summary) - exercise-solutions.ipynb	./ch03
Ch 4: Implementing a GPT Model from Scratch	- ch04.ipynb - gpt.py (summary) - exercise-solutions.ipynb	./ch04
Ch 5: Pretraining on Unlabeled Data	- ch05.ipynb - gpt_train.py (summary) - gpt_generate.py (summary) - exercise-solutions.ipynb	./ch05
Ch 6: Finetuning for Text Classification	- ch06.ipynb - gpt_class_finetune.py - exercise-solutions.ipynb	./ch06
Ch 7: Finetuning to Follow Instructions	- ch07.ipynb - gpt_instruction_finetuning.py (summary) - ollama_evaluate.py (summary) - exercise-solutions.ipynb	./ch07
Appendix A: Introduction to PyTorch	- code-part1.ipynb - code-part2.ipynb - DDP-script.py - exercise-solutions.ipynb	./appendix-A
Appendix B: References and Further Reading	No code	./appendix-B
Appendix C: Exercise Solutions	- list of exercise solutions	./appendix-C
Appendix D: Adding Bells and Whistles to the Training Loop	- appendix-D.ipynb	./appendix-D
Appendix E: Parameter-efficient Finetuning with LoRA	- appendix-E.ipynb	./appendix-E

Build a Large Language Model (From Scratch)