hashable 1.2.0.x segfaults with ByteString on 32bit Linux, GHC 7.0.4

[kazu-yamamoto]

On 32bit Linux, hashing ByteString causes segfault:

% ghci -XOverloadedStrings

:m Data.Hashable
:m +Data.ByteString.Char8
hash ("/" :: ByteString)
zsh: segmentation fault ghci -XOverloadedStrings

This bug introduced some between 1.1.2.5 and 1.2.0.0. This bug does not exist on 64bit Linux.

[kazu-yamamoto]

The master branch still causes segfault. I'm using GHC 7.0.4 on a 32bit Linux machine.

This bug exists in compiled code, too. Actually, I found this bug as (complied) Mighty caused segfault when it received an HTTP request.

[kazu-yamamoto]

Both hashable 1.2.0.4 and the current master (33a7234) do not solve this problem.

[kazu-yamamoto]

The following code caues this segfalt:

siphash-sse2.c: k0 = _mm_loadl_epi64((__m128i*)(&ik0));

[kazu-yamamoto]

Since I could not found a way to make use of SSE2 on 32bit Linux, I just made a workaround for the 8byte alignment problem:

diff --git a/cbits/siphash.c b/cbits/siphash.c
index 34901b4..4b0c3d9 100644
--- a/cbits/siphash.c
+++ b/cbits/siphash.c
@@ -87,7 +87,7 @@ static inline u64 _siphash24(u64 k0, u64 k1, const u8 *str, si
     return _siphash(2, 4, k0, k1, str, len);
 }

-#if defined(__i386)
+#if defined(__i386) && (defined(_WIN64) || defined(__amd64__))
 # undef _siphash24

 static u64 (*_siphash24)(u64 k0, u64 k1, const u8 *, size_t);

Both on 64bit Linux and 32bit Linux, cabal tests are passed.

On 32bit Linux, (hash "/") results in -1811599090.
On 64bit Linux, it returns 8470997533827433892.
Is this correct behavior?

[kazu-yamamoto]

Note that I have not tested this on Windows.

[bos]

@kazu-yamamoto I cannot reproduce your crash with the original test case on 32-bit Linux or Windows.

Also, it's expected that hashing gives different results on 32-bit and 64-bit systems, since Ints (and hence the default salt) are different sizes.

[kazu-yamamoto]

I installed GHC 7.4.2 for this 32bit Linux today. It does not cause this problem. So, this problem is caused by combination of GHC 7.0.4 and 32bit Linux. (GHC 7.0.4 on 64bit Linux does not cause this problem.)

[bos]

Thanks, @kazu-yamamoto, that was the extra information I needed.

[kazu-yamamoto]

My 32bit Linux (Linux 2.6.18) does not have le64toh in endian.h. Compiling hashable displays the following warning:

cbits/siphash.c:52:0:
     warning: implicit declaration of function ‘le64toh’

And GHC cannot compile my test program:

/home/kazu/.cabal/lib/hashable-1.2.0.4/ghc-7.0.4/libHShashable-1.2.0.4.a(siphash.o): In function _siphash_chunk.clone.0':
siphash.c:(.text+0x380): undefined reference to `le64toh'
/home/kazu/.cabal/lib/hashable-1.2.0.4/ghc-7.0.4/libHShashable-1.2.0.4.a(siphash.o): In function `hashable_siphash':
siphash.c:(.text+0x1050): undefined reference to `le64toh'
/home/kazu/.cabal/lib/hashable-1.2.0.4/ghc-7.0.4/libHShashable-1.2.0.4.a(siphash.o): In function `plain_siphash24':
siphash.c:(.text+0x1b40): undefined reference to `le64toh'
collect2: ld returned 1 exit status

[bos]

Your userland on that machine is 4 or more years old - endian.h was added to glibc in 2009. You're welcome to send a patch, but I'm not going to do anything about this, I'm afraid.

[kazu-yamamoto]

FreeBSD does not have endian.h but has sys/endian.h including le64toh.
Mac does not have endian.h. And I cannot find le64toh anywhere.

To bury differences, may I introduce "configure"?

[kazu-yamamoto]

I sent a pull request to solve this problem without using configure. What do you think?

kazu-yamamoto@556796c

[bos]

Very similar to 1360f47, which I was writing at the same time.

[kazu-yamamoto]

I confirmed that the current master works well on:

• MacOS (Mountain Lion)
• 32bit Linux
• 64bit Linux
• FreeBSD

Thank you very much.