-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(#3379) - prefer readAsArrayBuffer to BinaryString #3379
Conversation
Note that spark-md5 supports binary strings and array buffers, but not blobs. |
Neat, looks like I found a bug in spark-md5. |
Welp, it'll take me awhile to debug spark-md5. Adding WIP to the title. |
Ah jeez, it's a bug in our |
f0e3bba
to
05a1ead
Compare
I have no evidence that this produces better performance, but my intuition tells me so. It just seems silly to convert a Blob to a binary string just for the purposes of MD5 hashing. Presumably ArrayBuffers are better at memory management.
@@ -7,7 +7,7 @@ var MD5_CHUNK_SIZE = 32768; | |||
|
|||
function sliceShim(arrayBuffer, begin, end) { | |||
if (typeof arrayBuffer.slice === 'function') { | |||
if (!begin) { | |||
if (!begin && !end) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^ this was the sliceShim
bug. if you pass in 0 for begin
and non-zero for end
, then it will just call slice()
, which is wrong.
05a1ead
to
5c7ec91
Compare
I have no evidence that this produces better performance, but my intuition tells me so. It just seems silly to convert a Blob to a binary string just for the purposes of MD5 hashing. Presumably ArrayBuffers are better at memory management.
OK, this commit should be good now. |
5c7ec91
to
a49d832
Compare
I have no evidence that this produces better performance, but my intuition tells me so. It just seems silly to convert a Blob to a binary string just for the purposes of MD5 hashing. Presumably ArrayBuffers are better at memory management.
We should have a performance test that indicates this is some type of improvement, I ran the performance test and it ddint indicate any improvement, nor would I expect it since we dont have any attachment tests |
To me it's obvious for a few reasons:
Although I realize the onus is on me to provide a performance test justifying this. I'll try to get one soon. |
Yeh, I agree its obviously better, but its similiar to previous improvements we need to get the infrastructure in place not as much for this patch, but for future improvements to be measurable. I am focussed on your android patch right now, once I have done that I will make another go at the improvements I plan for the performance suite |
a49d832
to
7208cde
Compare
I have no evidence that this produces better performance, but my intuition tells me so. It just seems silly to convert a Blob to a binary string just for the purposes of MD5 hashing. Presumably ArrayBuffers are better at memory management.
Added a basic attachment perf test. It actually shows that the arraybuffer method is slightly slower:
These might be flukes, though. I should probably add more iterations and try to get a finer granularity. Also, if the memory usage is improved, these numbers won't capture that. |
Just a jshint failure here, I would be fine merging this to get the code fix and at least the performance tests in there, +1 when green |
7208cde
to
50dd98a
Compare
I have no evidence that this produces better performance, but my intuition tells me so. It just seems silly to convert a Blob to a binary string just for the purposes of MD5 hashing. Presumably ArrayBuffers are better at memory management.
OK, fair enough! jshint should be fixed now. |
Failing in node |
Oh God, if I want this canvas stuff to work in Node, we're going to need the I dunno. Maybe I should just check a large image into source control and load that? |
or just make a bigass text file |
I have no evidence that this produces better performance, but my intuition tells me so. It just seems silly to convert a Blob to a binary string just for the purposes of MD5 hashing. Presumably ArrayBuffers are better at memory management.
50dd98a
to
bb975fa
Compare
OK, I think we have a pretty decent test now. Just directly makes some Buffers/Blobs of arbitrary size. Makes more sense than my fancy canvas stuff. |
I like this test much better, a lot simpler |
I have no evidence that this produces better performance, but my intuition tells me so. It just seems silly to convert a Blob to a binary string just for the purposes of MD5 hashing. Presumably ArrayBuffers are better at memory management.
Merged in 98a1623 |
I have no evidence that this produces better
performance, but my intuition tells me so.
It just seems silly to convert a Blob to a binary string
just for the purposes of MD5 hashing. Presumably
ArrayBuffers are better at memory management.