jaydeepthik / Nano-GPT Public

Notifications You must be signed in to change notification settings
Fork 0
Star 4

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

4 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 11 Commits
data		data
GPT-raw.ipynb		GPT-raw.ipynb
README.md		README.md
bigram.py		bigram.py
gpt.py		gpt.py

Repository files navigation

Nano-GPT : Decoder only Transformer

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

Features

Multi-headed self attention
Layer Norm layers
Skip connections
Feed Forward layer

About

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

transformers pytorch gpt multihead-attention pytorch-implementation

Report repository

Releases

No releases published

Packages

No packages published

Languages