Key Error after fine tuning with gpt2

[thatfilmguy]

I'm getting the following error after running sample.py to generate text:

Traceback (most recent call last):
File "/Users/codypyper/nanoGPT-master/sample.py", line 88, in
print(decode(y[0].tolist()))
File "/Users/codypyper/nanoGPT-master/sample.py", line 68, in
decode = lambda l: ''.join([itos[i] for i in l])
File "/Users/codypyper/nanoGPT-master/sample.py", line 68, in
decode = lambda l: ''.join([itos[i] for i in l])
KeyError: 271

I trained the model originally on a custom dataset that I prepared. That worked fine, created a usable checkpoint and generated text using sample.py perfectly.

I then fine tuned my model by running:

python3 train.py config/finetune_project.py --device=mps --compile=False --out_dir=out-project

That step works fine, and the checkpoint was updated. But then when I run a sample.py with the updated checkpoint, i get the error mentioned earlier, "KeyError: 271"

Running on a
MacBook Pro (16-inch, 2021)
OS 12.5.1
Apple M1 Max
64GB Ram

Thanks for the help

[nirvitarka]

I got the same error when the prompt I gave had a question mark ("?"). It worked fine after I remove that "?"

[thatfilmguy]

I got it working. I ran the wrong prepare.py originally. I copied the prepare.py by copying the data/shakespeare-chars folder instead of the data/shakespeare. Thanks nirvitarka for the comment btw, good to know about the '?'

[GalMoore]

Thanks @thatfilmguy - had the same issue and same resolution :)