Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't train the model? #12

Closed
fdujay opened this issue Aug 31, 2019 · 3 comments
Closed

Can't train the model? #12

fdujay opened this issue Aug 31, 2019 · 3 comments

Comments

@fdujay
Copy link

fdujay commented Aug 31, 2019

(base) davis@davis-MS-7B17:~/Network/EMANet-master$ python train.py
2019-08-31 13:50:14,703 - INFO - set log dir as ./logdir
2019-08-31 13:50:14,703 - INFO - set model dir as ./models
2019-08-31 13:50:17,131 - ERROR - No checkpoint ./models/latest.pth!

The Training step is stopped, so I have to Keyboard Interrupt it...
Does anybody know how to solve it?

@fdujay fdujay changed the title AssertionError: Invalid device id Can't train the model? Aug 31, 2019
@fdujay
Copy link
Author

fdujay commented Aug 31, 2019

I have met the other error:
v = obj.cuda(dev, async=True)
^
SyntaxError: invalid syntax

Solving it by exchanging "async" by "non_blocking", because 'async' is a reserved word in Python >= 3.7

@XiaLiPKU
Copy link
Owner

XiaLiPKU commented Sep 4, 2019

(base) davis@davis-MS-7B17:~/Network/EMANet-master$ python train.py
2019-08-31 13:50:14,703 - INFO - set log dir as ./logdir
2019-08-31 13:50:14,703 - INFO - set model dir as ./models
2019-08-31 13:50:17,131 - ERROR - No checkpoint ./models/latest.pth!

The Training step is stopped, so I have to Keyboard Interrupt it...
Does anybody know how to solve it?

As the GPUs are loading model sequentially, you may need to wait for around 20s, and then the training process will start.

@XiaLiPKU
Copy link
Owner

XiaLiPKU commented Sep 4, 2019

I have met the other error:
v = obj.cuda(dev, async=True)
^
SyntaxError: invalid syntax

Solving it by exchanging "async" by "non_blocking", because 'async' is a reserved word in Python >= 3.7

v = obj.cuda(dev, async=True) is not included in my repo, and I haven't seed similar problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants