-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug on 32 bit systems? #56
Comments
I reproduced this with ghc-9.0 as well (i.e., use |
Nothing special here about
|
Hi John, I deleted all of my 32 bit code because I don't want to support it anymore. Do you have a dire need for it? Also, what version of |
Looking into it, I'd wager the memory boundaries for
A little less sound. I can have a fix out rather quickly, but I'd need test data. Can you provide the offending image so i can add it as a regression? I can just add that to the test suite and support i386 in CI without adding my word32-specific code back into the repo |
The Docker container above gives you everything you need to reproduce the problem. Any medium sized binary will work (I just use the As noted above, I've been able to reproduce problems (but not deterministically) with even fairly short bytestring literals. There is no urgency on my end, because I already switched back to using base64-bytestring in pandoc. |
If performance isn't your bottleneck, that's certainly a solution. It uses the worst-case-in-every-case approach. That said, on the branch for #57 I can't get it to trigger, even with multiple 32-bit CI attempts, so maybe the fix diagnosis was spot on? I'll need a regression tho, so if you end up with something that triggers it more deterministically than not, I'm all ears. I could also just attempt to encode/decode something against |
With whoami, it never failed to segfault for me. I only got indeterministic results for very short bytestrings (e.g. the first 20 bytes). Are there benchmarks of base64 vs bytestring-base64? I may go back if the problem is fixed. |
Okay - thanks i'll focus on that and see what i can dredge up. My guess is that when peeking a 64 byte word off, we cross some memory barrier somewhere that gets tripped in 32 bit systems when there are fewer than 2 words left in the array. The fix should target that case, since prior to the fix, we were looking only if there were at least six bytes left, not necessarily a full word, and GHC was always lax enough to say "if we attempt to read a full word and it's partial, fill with
Benchmarks exist here, and do compare Here's a sample of all recent output: https://github.com/emilypi/base64/blob/72cfd854ee3ba394e6dd7cfa0473d0fe542bf8ad/output.html. The long short of it is that ![]() |
OK, thanks, that's helpful. By the way, the initial image I tested with is |
After discussion with @kozross, I'm pretty sure the issue is as described. Merging and releasing |
I don't see a new release yet? |
Got caught up in the holidays and forgot. It's up now: https://hackage.haskell.org/package/base64-1.0 |
Actually, keeping this open just to confirm. |
Don't know if that's related, but we've been hit by SIGBUS/unaligned access on 32bit armv7a device 😟 |
@dpwiz could you give more about the details? My suspicion would immediately be the encoding loop in that situation, but that's probably because we use 64-bit intrinsics in that particular hotloop. This library dropped support for 32 bit arches a few years ago. |
Oops... Nvm then (: |
I'd still like to figure out how to solve it 😄 |
Well, that was some android mobile running 32bit/v7 chip. Apparently those can't tolerate unaligned reads, so the app got killed. |
I'll see if I can work it out - thanks for the lead @dpwiz :) |
Debian packaging ran into a problem with pandoc on i386, described at jgm/pandoc#9233. I think I've traced it to base64. Here is a Dockerfile that reproduces the segfault:
The text was updated successfully, but these errors were encountered: