Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upOutput of batch doesn't contain the output of a subet of a batch #31919
Comments
This comment has been minimized.
This comment has been minimized.
|
you should check that the difference is in an eps. precision is a real issue. |
This comment has been minimized.
This comment has been minimized.
|
@SsnL are you saying that the issue is due to floating point precision issues? I am wondering how come this is always true then? f(batch)[:batch] == f(batch[:batch]) |
This comment has been minimized.
This comment has been minimized.
|
@praateekmahajan Also, the general assumption that f(batch)[:k] == f(batch[:k])is always true, is not correct. Take batch norm layers as an example. |
This comment has been minimized.
This comment has been minimized.
|
Thanks @ptrblck, that's a good mention about batch norm layers. Didn't think about them. Not sure what you mean by
Thank you also for your suggestion on the It's still not entirely clear to me that we have some precision errors when using comparing a set with a subset of another set, and no errors when comparing two complete sets. Looks like when we compare two complete sets, there should be a higher chance of precision error, because there are more floats involved == i.e higher chance of encountering a floating point precision error |
This comment has been minimized.
This comment has been minimized.
|
In the future, please ask questions on our forums, discuss.pytorch.org. We like to keep our issue tracker filled with issues and feature requests. |
Might be an obvious question to some, but I am really struggling with understanding what could go wrong.
If we pass a batch input to a function, shouldn't the output of the batch also have the output of the subset of a batch?
Programmatically what I mean is
f(batch)[:k] == f(batch[:k])Shouldn't this be always true?
I try to reproduce the bug where
fis aLinearlayer without bias (and also the barebonematmuloperation).I can imagine someone saying that it's not safe to compare floats.
But then how come this is always true that :
f(batch)[:batch] == f(batch[:batch])Wrote a function here to reproduce the error