Can lightseq freeze the weight of gpt2 model?

[YINGPENGZH]

The following code is part of official example, My question is this line : generated_ids = model.sample(inputs) , will my model's weight will change with the inference job work for long time? Should I use some config to fix the weight of my model, like model.eval() in transformers.

[Taka152]

Could you give me a code snippet to reproduce this error?