Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrapping pytorch operations - grid_sample #47

Closed
ahyunSeo opened this issue Aug 31, 2021 · 4 comments
Closed

wrapping pytorch operations - grid_sample #47

ahyunSeo opened this issue Aug 31, 2021 · 4 comments

Comments

@ahyunSeo
Copy link
Contributor

ahyunSeo commented Aug 31, 2021

Hello,

Thank you for your nice work!

I'm about to use the e2cnn based network for my project.
I need to use the method grid_sample in PyTorch and its underlying implementation is just some interpolation.
https://pytorch.org/docs/stable/generated/torch.nn.functional.grid_sample.html?highlight=grid_sample#torch.nn.functional.grid_sample
I noticed the upsample in this repo simply computes the value and reassigns the output type.
https://github.com/QUVA-Lab/e2cnn/blob/master/e2cnn/nn/modules/r2upsampling.py#L86
I think I can do exactly like this with grid_sample.
Please let me know if there will be any issues with this.

Ahyun

@Gabri95
Copy link
Collaborator

Gabri95 commented Sep 2, 2021

Hi @ahyunSeo

What do you need grid-sample for?

Wrapping it in an EquivariantModule (as done in r2upsampling) is indeed not a problem from an implementation point of view.
However, grid_sample can perform some arbitrary deformation of the input, which means it is not generally equivariant and, therefore, should not be implemented in a subclass of EquivariantModule.

Are you maybe assuming some specific grid which guarantees equivariance?

Best,
Gabriele

@ahyunSeo
Copy link
Contributor Author

ahyunSeo commented Sep 2, 2021

Hi @Gabri95

You made a fair point.
I was about to sample a circular/polar grid for each pixel, where each pixel becomes its center.
So each pixel will be assigned with # angle of grid points.
Then, I wanted to compute correlation values between the grid points per pixel for the further pipeline.
Maybe I should move the feature tensor to a non-geometric tensor first.

Regards,
Ahyun

@Gabri95
Copy link
Collaborator

Gabri95 commented Sep 3, 2021

Hi @ahyunSeo ,

If the operation you are trying to implement is equivariant you can just wrap it in an EquivariantModule as I did for r2upsampling.
In the forward pass, you just extract the 'tensor' from the GeometricTensor input, process this tensor is an equivariant way and finally wrap the output in a new GeometricTensor.

You can precompute the grid in the init of the module such that you can ensure the grid allows equivariance.

Let me know if this helps

Gabriele

@ahyunSeo
Copy link
Contributor Author

ahyunSeo commented Sep 3, 2021

Thanks a lot!

@ahyunSeo ahyunSeo closed this as completed Sep 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants