Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Monte Carlo dropout does not work with pytorch-lightning >=2.2.0 #2300

Closed
eschibli opened this issue Apr 3, 2024 · 0 comments · Fixed by #2312
Closed

[BUG] Monte Carlo dropout does not work with pytorch-lightning >=2.2.0 #2300

eschibli opened this issue Apr 3, 2024 · 0 comments · Fixed by #2312
Labels
bug Something isn't working
Projects

Comments

@eschibli
Copy link
Contributor

eschibli commented Apr 3, 2024

Monte Carlo dropout appears to silently fail with the current release of pytorch-lightning.

To Reproduce
See this colab workbook. TCNModel instances with different dropout keywords produce identical CV results and inferences if given the same random_state, even with dropout set to 0.99, which should catastrophically underfit. I was able to reproduce this behaviour on my laptop using this python environment obtained by running pip install darts in a fresh Python 3.11 venv.

Expected behavior
The dropout keyword should impact model results. Extremely high dropout should result in severe underfitting. I was able to obtain this expected behavior in this environment by downgrading pytorch-lightning to 2.1.2. This can also be observed in colab

@eschibli eschibli added bug Something isn't working triage Issue waiting for triaging labels Apr 3, 2024
@madtoinou madtoinou removed the triage Issue waiting for triaging label Apr 4, 2024
@madtoinou madtoinou added this to To do in darts via automation Apr 4, 2024
@eschibli eschibli changed the title [BUG] Monte Carlo dropout does not work with pytorch-lightning 2.2.1 [BUG] Monte Carlo dropout does not work with pytorch-lightning >=2.2.0 Apr 8, 2024
@dennisbader dennisbader mentioned this issue Apr 10, 2024
3 tasks
darts automation moved this from To do to Done Apr 11, 2024
@dennisbader dennisbader moved this from Done to Released in darts Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
darts
Released
Development

Successfully merging a pull request may close this issue.

2 participants