Self-supervised learning - BYOL implementation #1305
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Scitator
previously requested changes
Oct 4, 2021
| source_model_key: key to the source data inside `runner.model` | ||
| tau: smoothing parameter `target * (1.0 - tau) + source * tau` | ||
| """ | ||
| super().__init__(order=CallbackOrder.Metric) |
There was a problem hiding this comment.
From my understanding, it should be on the same level as Criterion (change of the model), but maybe you have another point.
| # flake8: noqa | ||
| import argparse | ||
|
|
||
| from common import add_arguments, ContrastiveModel, datasets |
There was a problem hiding this comment.
could we use local naming?
like from .common?
Scitator
previously requested changes
Oct 4, 2021
Co-authored-by: Sergey Kolesnikov <scitator@gmail.com>
… into byol-implemention
… into byol-implemention
Scitator
previously requested changes
Oct 4, 2021
catalyst/runners/self_supervised.py
Outdated
Comment on lines
175
to
178
| if isinstance(self.model, (collections.Mapping, nn.ModuleDict)): | ||
| encoders = [(encoder_name, self.model[encoder_name]) for encoder_name in self.model] | ||
| else: | ||
| encoders = [("", self.model)] |
There was a problem hiding this comment.
let's move it to on_stage_start
so we could set up some internal flag like is_kv_model once per stage rather than every batch
| # flake8: noqa | ||
| import argparse | ||
|
|
||
| from common import add_arguments, ContrastiveModel, datasets |
Scitator
reviewed
Oct 4, 2021
catalyst/runners/self_supervised.py
Outdated
| @@ -13,6 +16,7 @@ class ISelfSupervisedRunner(IRunner): | |||
| augemention_prefix: key for ``runner.batch`` to sample augumentions | |||
| projection_prefix: key for ``runner.batch`` to store model projection | |||
| embedding_prefix: key for `runner.batch`` to store model embeddings | |||
| encoders: encoder keys inside `model` | |||
Scitator
previously approved these changes
Oct 4, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before submitting (checklist)
catalyst-make-codestyle -l 99 && catalyst-check-codestyle -l 99(pip install -U catalyst-codestyle).make check-docs?pytest .?latestandminimalrequirements?Description
Bootstrap Your Own Latent (BYOL) is an approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view.
For this pipeline, we need to implement a callback for updating the target network from the online network (SoftUpdateCallback) and add a multi-encoder strategy in SelfSupervisedRunner.
Related Issue
Type of Change
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
FAQ
Please review the FAQ before submitting an issue: