Package up useful changes from my experimentaiton branch #37
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I hypothesize that blow-up in SAM is connected to linearly unstable modes of the neural network. This commit introduces code for computing the jacobian and it's eigenvalues in an autodiff friendly way. Also, the weak temperature gradient in the tropics significantly alters what the unstable modes will look like, so I include code that penalizes unstable WTG modes. These changs required some additions to the Batch class in the timestepper module.
Previously the dataset class defined a sample as the whole time series from an indididual location, but this does not allow enough separate training steps per batch. This code allows dividing the time series into equal non-overallaping chunks.
All the training methods are now based timestepper.Batch objects rather than having to create them on their own. This required some changes in the loss functions. I also added plots for the moisture budget imbalance.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The last weeks experimentation branch got too crazy. I am repacking the useful parts to be merged into master.