Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error NBEATS : Epoch[0] gave NaN loss and it will be ignored #26

Open
mdancho84 opened this issue Jun 13, 2021 · 1 comment
Open

Error NBEATS : Epoch[0] gave NaN loss and it will be ignored #26

mdancho84 opened this issue Jun 13, 2021 · 1 comment

Comments

@mdancho84
Copy link
Contributor

I'm trying to run a version with the "Alternative Padding Strategy" suggested in 18.1. Everything has worked well until Model 6 NBEATS ensemble. I've tried adjusting the nbeats() arguments, but the workflow still crashes. Have you seen the error below before? I found a similar warning online for MQCNNEstimator, not for NBEATSEstimator or NBEATSEnsembleEstimator.

> wflw_fit_nbeats_6_2 % 
+    add_model(model_spec_6_nbeats) %>% 
+    add_recipe(recipe_spec_gluon_2) %>% 
+    fit(training(splits_2))
  0%|          | 0/35 [00:00, ?it/s]WARNING:gluonts.trainer:Batch [2] of Epoch[0] gave NaN loss and it will be ignored
@vidarsumo
Copy link

I've had this issue with DeepAR and then it was due to exploding gradients and I had to adjust the clip_gradient. I don't know the exact architecture of N-BEATS, but could this also be the case here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants