Some question about the code

[liuxiaozhu01]

Hi! Thanks for your great work!

Im a little confused about the implementation. Your simple and efficient work only requires once forward caculate to get the activation of each layer. This line seems means that the forward caculate is excuted in sparse network, which also means the input of next layer is caculated by the current sparse layer because current layer is already masked.

Im wondering whether the "masked forward" is necessary, and I notice that if this operation is canceled, the result can be better in some conditions.

[Eric-mingjie]

We do not examine this closely in the paper, which we use to inherit the setup of SparseGPT. This does seem to be helpful for the LLaMA and LLaMA2 models we examined in the paper.

[liuxiaozhu01]

I have read your paper carefully again. And i notice that this operation is mentioned as this "Given a pretrained LLM, we compute our pruning metric from the initial to the final layers of the network. After pruning a preceding layer, the subsequent layer receives updated input activations, based on which its pruning metrics will be computed."

Do you mean that the "masked forward" is helpful to LLaMA and LLaMA2 models?

And thanks for your reply! I think I need to do more further examination.