Refactor models and losses #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. model.call() does not return inputs anymore, which was confusing and
potentially misleading
2. Make inputs to loss functions explicit (no getters with magic
numbers)
Before: all loss functions were receiving everything, including
unnecessary data;
Now: loss functions only get what they need as inputs, loss terms
are explicitely written in models
3. Add neural and behavioral scales: these parameters should make the
model invariant to the scale of data inputs.
4. Add 'full' and 'causal' decoders
5. Fix TNDM model saving: now decoder dimensions are saved
Conditions are needed for the new R2... there is no need for them in the main repo at the moment.
|
Reviewing now |
|
Added a new param, soft_max_min_poisson_log_firing_rate, to both LFADS and TNDM. Hopefully, that doesn't break anything. Now we can adjust that if it is an issue. The PR looks good! I would add the new likelihoods and masks in a separate PR as we discussed. |
Looks good to me. Unit tests are also doing fine. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This refactoring is necessary if we want to work with different likelihoods for neural and behavioral data.
Before this commit: every loss function was receiving everything (all initial conditions, latents, rates, etc.), including unnecessary data;
Now: loss functions only get what they need as inputs, loss terms are explicitly written in models