Different results for output after loading the weights of GPT2

[Monrother]

I cloned the repo, and the output after I loaded the GPT 2 weights were different than the one shown in the book.

For "Every effort moves you", my local output is
Every effort moves you toward an equal share for each vote plus half. Inequality is often not an accurate representation of human worth; to know the

which I'm not sure what it is talking about.

And from the book, it is:
Every effort moves you toward finding an ideal new way to practice something!
What makes us want to be on top of that?

Which makes more sense.

Is this expected?

[casinca]

I presume you ran the notebook on Win/Linux? your output isn't abnormal, I'm getting the same, if that reassures you and you'd expect the same output from the book on Mac.
Even though it's not directly linked to dropout since it's disabled for inference, I believe it's still related to that MKL issue: pytorch/pytorch#121595

As for the quality of the trajectory, it's just luck. You could change the seed and try to get something better. Even if weights were not properly loaded and failed silently, it would have been much worse.

On top of that, these hparams temp>1 and high topk, increase the entropy of the probability distributions. For larger models, we could say it increases "creativity" but here for a small model, it'll likely increase nonsenses too, compared to greedy decoding.

[Monrother]

Thanks, @casinca. Yes I'm using Windows. I tested when using the same code with the repo so I had the feeling that it's because of something in the library. Thanks for sharing the MKL issue.