Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move model to device after pytorch interpolation #2525

Merged
merged 2 commits into from Jan 27, 2024

Conversation

joeyballentine
Copy link
Member

Fixes the bug mentioned by metin on discord

@joeyballentine joeyballentine changed the title Move model to device after interp Move model to device after pytorch interpoolation Jan 27, 2024
@joeyballentine joeyballentine changed the title Move model to device after pytorch interpoolation Move model to device after pytorch interpolation Jan 27, 2024
@RunDevelopment
Copy link
Member

... Why does this solve the bug?

@joeyballentine
Copy link
Member Author

joeyballentine commented Jan 27, 2024

Since the load model node moves the model to gpu, if you did an interp into another interp, where the second interp comes from a load model node (so model 1 and model 2 into interp 1, and interp 1 and model 3 into interp 2), then one model would be on the gpu and one would be on cpu. This fixes that and makes it consistent.

Alternatively, perhaps we could change the model input's enforce method to always move it to the specified device?

@RunDevelopment
Copy link
Member

Okay, I just looked at the discord messages. This is not a fix.

The issue is that the node currently can't interpolate 2 models when they are on different devices. So you tried to make sure that this node puts the new model it returns on the same device as other nodes.

Why is this not a fix? You didn't solve the issue that this node cannot interpolate 2 models when they are on different devices. The cause of the issue still remains and there will be a bug report again in the future. For example, like this:

  1. Add a Load model node and select a model.
  2. Go to settings and change PyTorch from GPU to CPU (or vise versa).
  3. Add another Load model node.
  4. Setup the chain to interpolate both models are run it.
  5. Same error.

Instead of assuming that models will always be on the same device, just move both models to the same device.

@RunDevelopment
Copy link
Member

Alternatively, perhaps we could change the model input's enforce method to always move it to the specified device?

This would require inputs to have access to the current node execution context. Not ideal, since this would prevent optimizations in the future.

@joeyballentine joeyballentine merged commit ce8c46b into main Jan 27, 2024
14 checks passed
@joeyballentine joeyballentine deleted the fix-inter-from-interp branch January 27, 2024 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants