Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exiting early doesn't save latest training state #12

Closed
awused opened this issue Oct 19, 2023 · 2 comments
Closed

Exiting early doesn't save latest training state #12

awused opened this issue Oct 19, 2023 · 2 comments
Labels
bug Something isn't working enhancement New feature or request solved issue has been solved

Comments

@awused
Copy link

awused commented Oct 19, 2023

If you exit early it saves the latest model (net_g_latest and net_d_latest) but not the latest training state, so when you resume training it restarts from the latest regular checkpoint.
@neosr-project
Copy link
Owner

Hi, sorry for the late reply. This is a known issue that, unfornatelly, is more complicated to solve than it seems. Hopefully I can take a look at it in the future. Feel free to send PRs regarding the issue.

@neosr-project
Copy link
Owner

Added a section to the wiki for known issues.

@neosr-project neosr-project added bug Something isn't working enhancement New feature or request labels Dec 19, 2023
@neosr-project neosr-project added the solved issue has been solved label Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working enhancement New feature or request solved issue has been solved
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@awused @neosr-project