Skip to content

therealoliver/Deepdive-llama3-from-scratch

Error
Looks like something went wrong!

About

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

ko-fi.com/therealoliver

Topics

tokenizer inference transformer attention llama gpt mask language-model attention-mechanism rms rope residuals multi-head-attention kv-cache positional-encoding llms rotary-position-encoding rms-norm swiglu llm-configuration

Resources

Readme

License

MIT license
Activity

Stars

596 stars

Watchers

4 watching

Forks

47 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 2

Languages