Merge pull request #1 from michuhu/minor-fixes

Update README.md

Update CITATION.cff

Create CITATION.cff

Update README.md

Output linear transformation weights initialization fix and heads ite…

Increase default hparams a bit

Minor updates, trying to align my code with Karpathy's, maybe it'll i…

Gradient clipping, detect anomaly, use pad tokens in data loader, shu…

Fixing dimensionalities issues

Watched Karpathy's GPT video and identified issues with attention com…

Use masked_fill for causal masking

Add dropout to attention head and verify tokenizer

DataLoader suffle default to False

Code reformat

Minor device changes

Update README.md

Add train and run README

Show delimiter when running inference

Top k and temperature to prevent the same token repeating

Inference code

Add inference code (not yet tested), improve default hyperparameters

Use right device

requirements.txt

Add checkpointing

Fixed dimensionality issues, seems to be ready for the training

Working on training loop and debuggin index issue in embeddings

PE fix

Fixing statically found issues

Rewrite forward() and model structure for GPT and TransformerBlock