Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nglod does not save the weights+model #160

Closed
ttsesm opened this issue Jul 5, 2023 · 1 comment
Closed

nglod does not save the weights+model #160

ttsesm opened this issue Jul 5, 2023 · 1 comment

Comments

@ttsesm
Copy link

ttsesm commented Jul 5, 2023

I am testing the app/nglod example in an interactive mode. While in the config file it is set to save the model every 5 epochs:

trainer:
  # Base Trainer config
  exp_name: "nglod-octree"  # Name of the experiment: a unique id to use for logging, model names, etc.
  mode: 'train'             # Choices: 'train', 'validate'
  max_epochs: 10            # Number of epochs to run the training.
  save_every: 5             # Saves the optimized model every N epochs
  save_as_new: False        # If True, will save the model as a new file every time the model is saved
  model_format: 'full'      # Format to save the model: 'full' (weights+model) or 'state_dict'
  render_every: 5           # Renders an image of the neural field every N epochs
  valid_every: 10           # Runs validation every N epochs
  enable_amp: True          # If enabled, the step() training function will use mixed precision.
  profile_nvtx: True        # If enabled, nvtx markers will be emitted by torch for profiling.
  grid_lr_weight: 1.0    # Learning rate weighting applied only for the grid parameters (contain "grid" in their name)

when I check on the log folder there is nothing saved. Any idea why?

@ttsesm
Copy link
Author

ttsesm commented Jul 5, 2023

ok, had to comment the following line

cfg.trainer.save_every = -1
for making it work on interactive mode

@ttsesm ttsesm closed this as completed Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant