-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
integrations: move callbacks/loggers to frameworks #221
Comments
Another option would be to try to maintain integrations compatible across different versions by adding conditionals in our code. For example (#219): if self.model_file:
if catalyst.version <= 0.22.0:
checkpoint = runner.engine.pack_checkpoint(
model=runner.model,
criterion=runner.criterion,
optimizer=runner.optimizer,
scheduler=runner.scheduler,
)
else:
# code compatible with >= 0.22.0
. . . This could potentially become hard to maintain |
We should get rid of integrations and transfer to the framework itself when possible. As mentioned in the first point of #223 |
We don't own ML framework dependencies and it is ML framework responsability to run the safety checks. We will eventually move framework integrations to the framework itself #221
Originally Posted by @pared |
We don't own ML framework dependencies and it is ML framework responsability to run the safety checks. We will eventually move framework integrations to the framework itself #221
Would this require a major version bump? I think it's time we need to think about doing this. |
🤔 Unless we do some hacky stuff to keep the imports working with the same syntax as today, yes (probably not worth the effort) |
Opened huggingface/transformers#27352 for hf transformers |
Opened huggingface/accelerate#2139 for hf accelerate |
Lightning no longer accepts new callbacks 😦. Wish I had seen this before I wrote the PR 😅 :
Lightning-AI/pytorch-lightning#14137 (comment) Once the huggingface callbacks are released and documented, we can lower the priority on this issue. |
@daavoo I think that this is quite a challenge for us to decide.
On one hand we don't want to restrict the dependencies, because that could interfere with users workspace, and
dvclive
is probably not that important to force user to install particular version of the DL framework.On the other hand, we need to maintain our project and the wide range of supported frameworks makes it hard to be able to know which versions of particular frameworks we support. Solution to that would be restricting the supported versions, but that contradicts with 1. We can discuss this on our next meeting.
Originally posted by @pared in #220 (comment)
The text was updated successfully, but these errors were encountered: