A question about training weights of embedding

[AriKing11]

I have a question about the training weights of embedding. I used my own datasets to process stage 1 (which includes tuning the embedding weights of new graph tokens, e.g. DEFAULT_GRAPH_TOKEN = ""), but the weights became Nan instantly, I don't know why. Thanks for your patience.

[tjb-tech]

I have a question about the training weights of embedding. I used my own datasets to process stage 1 (which includes tuning the embedding weights of new graph tokens, e.g. DEFAULT_GRAPH_TOKEN = ""), but the weights became Nan instantly, I don't know why. Thanks for your patience.

Thanks for you interests! May I ask details of your error? Do the weights become Nan or loss become Nan?

[msy0513]

I have a question about the training weights of embedding. I used my own datasets to process stage 1 (which includes tuning the embedding weights of new graph tokens, e.g. DEFAULT_GRAPH_TOKEN = ""), but the weights became Nan instantly, I don't know why. Thanks for your patience.

Thanks for you interests! May I ask details of your error? Do the weights become Nan or loss become Nan?

我更换数据后再stage1和stage2都得到了train_loss=nan,这是正常情况么？该怎么解决这个问题呢？