Wrong SHA-256 for files bigger then 512 MB

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Read a file of 536.870.912 bytes or bigger with 
FileReaderSync.readAsArrayBuffer() in chunks of 1MB;
2. For each chunk, run the "CryptoJS.algo.SHA256.update" method (the entire 
code is attached).
3. The hash is wrong.

What is the expected output? What do you see instead?
Expected: 3a6e761d5033b09bd41528b23501c906ebdf8a30bcd4ee35bb4d22d5beae0fb8
Result: 8514390ffc132aa5cc1d1234d629faae4675016f7ed632bc6755fc987432708c


What version of the product are you using? On what operating system?
3.0.2 - Windows 7 (Firefox 10 and Chrome 21)

Please provide any additional information below.

The hash for files of size until 536.870.911 bytes are right.

It's probably a problem of converting a C code to JS. In C, the "int" has 32 
bits so it overflows on 4.294.967.296 bits (536.870.912 * 8). JS doesn't have 
this problem.

We could to fix this problem in this lib [1]. In the "sha256_final" function, 
he uses the count array to control the overflow (count[0] for the first bits 
and count[1] for the overflow). We forced the overflow with the 536.870.912 
bytes file by setting "count[0] = 0" and "count[1] = 1" and it worked.

We tried to do the same thing with crypto-js (which is a faster lib) but we 
couldn't understand the code (it's not your fault - it's ours).

[1] http://www.bichlmeier.info/sha256.js

Original issue reported on code.google.com by rcsilv...@gmail.com on 4 Sep 2012 at 10:40

Attachments:

• hashworker.js

[GoogleCodeExporter]

You're right. Hash functions like SHA and MD5 append the message length in bits 
to the end of the message. By definition, that length is a 64-bit number. But 
several JS implementations, including CryptoJS, counted using only a 32-bit 
number. That decision was partly for simplicity, partly because it was hard to 
imagine using JavaScript to hash 512 MB or more. Nonetheless, it's a flaw, and 
I'll have it corrected.

Original comment by Jeff.Mott.OR on 4 Sep 2012 at 11:11

• Changed state: Accepted

[GoogleCodeExporter]

My colegue found a solution. Change the line 135 of the sha256.js file from:

dataWords[(((nBitsLeft + 64) >>> 9) << 4) + 15] = nBitsTotal;

to:

dataWords[(((nBitsLeft + 64) >>> 9) << 4) + 14] = Math.floor(nBitsTotal / 
4294967296);
dataWords[(((nBitsLeft + 64) >>> 9) << 4) + 15] = nBitsTotal & 0xFFFFFFFF;

(Sorry I don't have the diff file...)

Original comment by rcsilv...@gmail.com on 4 Sep 2012 at 11:18

[GoogleCodeExporter]

Yes, that's definitely an improvement. JavaScript numbers are 64-bit floating 
point (aka double), so we get 53-bits before we start losing precision. Your 
colleague increased possible message lengths from 32 to 53-bits. I'll also look 
into a fix to get the full 64-bits.

Original comment by Jeff.Mott.OR on 5 Sep 2012 at 12:31

[GoogleCodeExporter]

I think I'm seeing this issue for SHA1 and MD5. 53bits doesn't sound to shabby. 
Not sure anyone wants to hash more than a petabyte in js anytime soon. I 
definitely don't need the full two exabytes ;-)

Original comment by hacker.s...@gmail.com on 27 Sep 2012 at 12:03

[GoogleCodeExporter]

Latest release includes your colleague's solution.

Original comment by Jeff.Mott.OR on 7 Jan 2013 at 1:57

• Changed state: Fixed

[GoogleCodeExporter]

Great! Many thanks!

Original comment by rcsilv...@gmail.com on 7 Jan 2013 at 3:48