-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Unconfirmed] Data corruption after recent upgrade #39
Comments
In WAL-G we observed some related data corruptions. Currently, we are hunting this too. |
Thanks for reporting! |
facebook/zstd#1300 may be related (don't know). |
In my case I cannot reproduce the problem with less than 2 MAXGOROUTINES, compressed size does not exceed few Gb. I still suspect that there might be something broken in WAL-G, but lz4 and lzma do not fail on similar tests (but this is not a proof, actually, just a hint...) |
Hi,
Our application stores and loads compressed data on disk using this library. We recently upgraded our application from 0727e17 (tag v1.3.0) to aebefd9 (tag v1.3.4).
After this library upgrade, there were some complaints from users. After generating some JSON data, compressing it, storing it, then later loading it and decompressing it, the data could no longer be parsed (e.g.
json.Unmarshal: invalid character '\x00' in string literal
).The issue occurred on (at least) both Windows and macOS, with both Go 1.9.7 and 1.10.3.
I assume it was caused by memory corruption in CGO data buffers.
Reverting this library back to tag v1.3.0 seems to have completely resolved the issue going-forward.
We're not yet able to reproduce the issue, and not all our users were affected (perhaps it's related to OS memory pressure?), but, just a heads-up that this library upgrade was implicated. Once we have an internal reproducer we may be able to bisect it (or hopefully, blame something else and disregard this entire issue).
The text was updated successfully, but these errors were encountered: