ModularTorchModel #3242
ModularTorchModel #3242
Conversation
| self.model = self.build_model() | ||
| self.model.to(self.device) | ||
|
|
||
| def load_from_modular(self, |
There was a problem hiding this comment.
I think it may be better to just call this load_from_pretrained and disable the mypy complaint? Feels like introducing a new API is more confusing than a slight mismatch in the call structure
There was a problem hiding this comment.
But we're still introducing a new API. We're just naming it the same as an old one.
|
It looks like some additional tests are failing as well. By comparison to #3241 which has 20 green, we only have 15 tests green suggesting there may be some breakages |
|
@rbharath I believe this is ready for a final review |
| raise NotImplementedError("Subclass must define the loss function") | ||
|
|
||
| def freeze_components(self, components: List[str]): | ||
| """Freezes or unfreezes the parameters of the specified components. |
There was a problem hiding this comment.
Slight edit, "unfreezes" should no longer be in this docstring
| self.model = self.build_model() | ||
| self.model.to(self.device) | ||
|
|
||
| def unfreeze_components(self, components: List[str]): |
There was a problem hiding this comment.
We should add a unit test for unfreeze_components
Pull Request Template
Description
This PR adds in a new abstract model, ModularTorchModel. The premise of ModularTorchModel is that we may want to take components from different models and combine them, or modify a model to work with a different task. This is useful for transfer learning where we may want to take an embedding from a pretrained model, or modify a model's prediction head to work with a pretraining task.
Type of change
Checklist
yapf -i <modified file>and check no errors (yapf version must be 0.32.0)mypy -p deepchemand check no errorsflake8 <modified file> --countand check no errorspython -m doctest <modified file>and check no errors