[Festure Request] Suggested enhancement for SpatialSoftArgmax2d

[vimalthilak]

@edgarriba : First of all, very nice repository!

I have a suggested enhancement that would help users implement recently published papers. I was inspecting the class that impelments softargmax2d

class SpatialSoftArgmax2d

and thought that you might want to add a temperature parameter for your class that a client can then use to specify the peakiness of their desired distribution.

[edgarriba]

@vimalthilak thanks for the suggestion. Yes, we can definitely do that. Will mark as a to do after the refactoring we are doing in library organization right now. Besides, any help is very welcomed :D

[edgarriba]

@vimalthilak looking into this after huger refactor. Welcome to kornia :D

That would be my suggestion for the signature:

def spatial_soft_argmax2d(
        input: torch.Tensor,
        temperature: Optional[Union[float, torch.Tensor]] = 1.0,
        normalized_coordinates: Optional[bool] = True) -> torch.Tensor

is guess one would like to pass a learned tensor of temperatures to be optimized.
Please share your thoughts on that before we go ahead with the implementation.

[bhack]

It could be also nice to have the dsntnn version of the SpatialSoftArgmax2d: https://github.com/anibali/dsntnn.
/cc @anibali

[vimalthilak]

@vimalthilak looking into this after huger refactor. Welcome to kornia :D

That would be my suggestion for the signature:

def spatial_soft_argmax2d(
        input: torch.Tensor,
        temperature: Optional[Union[float, torch.Tensor]] = 1.0,
        normalized_coordinates: Optional[bool] = True) -> torch.Tensor

is guess one would like to pass a learned tensor of temperatures to be optimized.
Please share your thoughts on that before we go ahead with the implementation.

This signature looks reasonable. It generalizes what I had in mind in terms of treating the temperature as a hyperparameter that's not learned but I think your use case should accommodate this usage as long as the client sets requires_grad=False.

[anibali]

@bhack Feel free to take as much as you would like from DSNTNN, I wouldn't mind if Kornia eventually rendered my little library obsolete.

[edgarriba]

closing this since solved in #166

[RSKothari]

Don't want to create a new issue for you so adding onto current feature request.

Request return of softmax (with temperature parameter incorporated) along with soft-argmax coordinates. This could be helpful for multitask applications.

[edgarriba]

@RSKothari thanks :)

[anibali]

@RSKothari If you look at spatial_soft_argmax2d, it's a really basic wrapper that calls two other functions which do all the work (dsnt.spatial_softmax_2d and dsnt.spatial_softargmax_2d). So, if you need the output of both, I'd recommend just calling both yourself instead of using the wrapper.

For example:

from kornia.geometry import dsnt

heatmaps = dsnt.spatial_softmax_2d(some_tensor, temperature=1.0)
coordinates = dsnt.spatial_softargmax_2d(heatmaps, normalized_coordinates=True)

[edgarriba]

@vimalthilak is this issue still relevant ? otherwise we can close it

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions, and happy coding day 😎