Repository navigation
Sharing a fuzzer for Burn #5660
SRaswan
started this conversation in
Show and tell
Replies: 1 comment
|
Thanks for sharing! This seems particularly useful for catching interactions between views, autodiff and more that isolated operator tests can miss. It could also help validate backend contracts, including consistent edge-case behavior such as NaN propagation. I could see something like this running periodically, maybe weekly with a time limit, against current Burn and available backends. Reproducible failures could then be minimized and added to our regression tests, broadening coverage without running the full fuzzer on every PR. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi! I wanted to just share a differential fuzzer for Burn (for both tensor operations and automatic differentiation) that I think the team might find interesting. This was tested against 0.20.1 a while ago and uncovered a few bugs. It randomly generates ASTs for Burn to compile and test against Burn's LibTorch backend.
https://github.com/SRaswan/burnmark
Here is a presentation we made about it.
One of the bugs it uncovered, for example, was a memory management bug within Burn’s LibTorch backend binding in 0.20.1 (that is now fixed in 0.22!).
The randomly generated function looked like:
And it reproduced:
The bug stems from how the FFI handles tensor cloning and memory views. In tensor frameworks, operations that modify a tensor’s view (such as a transpose) require a proper clone if the tensor is being modified in-place without altering the original underlying data.
Through our fuzzer’s automated forward and backward pass evaluations, we discovered that Burn’s LibTorch binding was incorrectly performing a shallow clone rather than a deep copy, despite the explicit clone() calls. When the transposed view was modified or used in subsequent graph operations, the shallow clone caused unintended memory sharing across the FFI boundary, leading to incorrect gradient calculations and memory corruption.
The resolution to this bug is what is in the code now. The backend can now safely detect when memory sharing exists and force a deep copy during in-place view modifications.
Just wanted to share this! I hope to also extend in the future with more rigorous testing against 0.22 and with more features (like higher-rank operations, more tensor operations, type-directed generation, etc.).
All reactions