Intermediate layer output

[bakszero]

Is it possible to use the intermediate layer outputs and generate text ignoring the layers on top?
Basically, I want to check quality of generations as we keep on adding more layers. What modifications in the src/sample.py script would I have to make for the same? Thanks.

[MrKrzYch00]

I did experiment once with changing amount of heads and layers. For 345M the optimal seems to be the model set amount. I think it was a multiply of 2 but I'm not sure anymore as I don't remember. Just overwrite the data read from model. Hint: search for hparams.n_ctx in the code and check model configuration file.

In my opinion, however, it's better to just play with temperature or logits (diversity).