add deepfloyd model #1993
add deepfloyd model #1993
Conversation
| grad_scaler, | ||
| ) -> torch.Tensor: | ||
| with torch.autocast(device_type="cuda", enabled=False): | ||
| image = F.interpolate(image.half(), (IMG_DIM, IMG_DIM), mode="bilinear", align_corners=False) |
There was a problem hiding this comment.
Does IMG_DIM always need to be 64? If not, we might want to add it as an argument.
There was a problem hiding this comment.
We're only using the first model of Deepfloyd for SDS, which is trained to generate 64x64 images. Not sure what the results would look like if we used a different image size though.
the other two models in the pipeline are the super resolution models that allow it to generate higher resolution images
There was a problem hiding this comment.
ok, feel free to ignore my comment
There was a problem hiding this comment.
I would make the filename just utils.py. I can imagine more generative util functions in the future, but probably few if any gradient specific utils.
|
i tried resolving the last 3 pyright errors but im unsure how to fix the typing for them |
nerfstudio/generative/deepfloyd.py
Outdated
| text_embeddings, | ||
| image, | ||
| guidance_scale, | ||
| grad_scaler, |
No description provided.