[Error] An pRuntimeError: invalid argument 5: k not in range for dimension at C:/w/b/windows/pytorch/aten/src\THC/generic/THCTensorTopK.cu:26ython exception is encountered at ...

[MRZHANG-1997]

Describe the error A clear and concise description of what the problem is. This question is caused by the top_choices_num which is from stage 2 of training, when I use my data of 37 category for training. This error will be solved when the top_choices_num is set 30. I try to understand this parameter, but I am failure by the code. I would be grateful if you would explain this parameter. Your work is very meaningful to solve the long tail problem, thank you for sharing.

[TonyLianLong]

This is part of the setting in EA step, where we want to give the neural network useful information about whether it should continue to the next step, as we make use of the only top k logits in the expert assignment. If your dataset has only 37 classes, then each prediction of an expert has only 37 logits, and setting k to a larger number does not make sense. This is what leads to the error in your code. You could try to play with different k. However, since your dataset has very few classes compared to the datasets that we use, I personally think the value of k will not affect your training results too much.

[MRZHANG-1997]

Thank you very much for your reply! Now, I get its role which is very helpful to understand the code.