-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
reduction=none broken #69
Comments
I have the same issue |
The reason this currently doesn't work is that the desired behavior for We could consider returning an list of tensors that correspond to each |
I think this would work for me, what I need is to have one aggregated loss per batch instead of aggregating over all dimensions. Let me know if I'm missing something! |
Perhaps the best way to achieve the desired behavior right now is to manage a set of
outputs
|
Great, I will do this instead. Thanks so much! |
I came up with another hacky way to do this while leveraging all the convenience of the
|
@csteinmetz1 Also, I noticed that for spectral convergence loss, despite the option `reduction=None, we nevertheless get a reduced result, because it has been implemented like this: return torch.norm(y_mag - x_mag, p="fro") / torch.norm(y_mag, p="fro") instead of return torch.norm(y_mag - x_mag, p="fro", dim=(-1, -2), keepdim=True) / torch.norm(y_mag, p="fro", dim=(-1, -2), keepdim=True) and noticing this also made me wonder if the denominator of the current implementation makes sense. I know that few other works like |
the denominator was averaged over all dimensions including the batch dimension, see comment by @egaznep in csteinmetz1#69
gives
What I'm really looking for is a per-instance reduction, which I can compute from reduction=none.
Anyway, reduction=none is not working out of the box, which is unfortunately a showstopper for me :(
The text was updated successfully, but these errors were encountered: