cuda error: an illegal memory access was encountered

[visonpon]

Hi @pureexe, thanks for your great work, but when I trained on m own datasets for a while, I got a Cuda error, when I change to train on only one GPU, it can train for a longer time, but can also trigger this error

the error messages like below:
train.py in forward cof=pt.repeat_interleave(cof,args.sublayers,0)
runtimeerror: an illegal memory access was encountered

But when I use the demo-room datasets to train, it seems the training phase is normal. I use the colmap to preprocess the datasets and get the hwf_cxxy.txt and poses_bounds.npy using the scripts you provided.

btw, when train on my own datasets, how to set the plane.txt? hope you can give some advice, thanks~

[pureexe]

Could you give the command that you use to run the train.py code? What config that you used?

Do you set up the environment with our environment.yml? If you use your own environment, what is your PyTorch version?

Could you give us your dataset? We will test it on our side.

Also, what is your machine spec? (GPU, CPU, RAM)

[visonpon]

my GPU is TITAN X,Cuda version is 10.1,drive version is 418.56,torch1.4.0. it seems that this error is related to my environment, but since trying to reinstall all the dependence is complex, I will try to debug this error and see if there are other ways to solve it, thanks~

[X-kimna]

I also encountered this problem，have you solved it？