Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: consolidate encoding parsing #29217

Merged
merged 2 commits into from Aug 28, 2019

Conversation

@mscdex
Copy link
Contributor

commented Aug 20, 2019

This consolidates encoding parsing for buffer at both the JS and C++ levels (only fill() still parses the encoding twice). Doing so not only cleans things up but also improves performance in some cases.

Various relevant benchmark results:

                                                                                                   confidence improvement accuracy (*)   (**)  (***)
 buffers/buffer-bytelength.js n=10000000 len=16 encoding='base64'                                          *      4.03 %       ±3.40% ±4.53% ±5.90%
 buffers/buffer-bytelength.js n=10000000 len=16 encoding='buffer'                                        ***     17.47 %       ±1.09% ±1.46% ±1.90%
 buffers/buffer-bytelength.js n=10000000 len=16 encoding='utf8'                                          ***     14.05 %       ±2.17% ±2.92% ±3.87%

 buffers/buffer-tostring.js n=10000000 len=64 args=0 encoding='ascii'                                    ***      1.05 %       ±0.58% ±0.78% ±1.03%
 buffers/buffer-tostring.js n=10000000 len=64 args=0 encoding='hex'                                      ***      2.09 %       ±0.76% ±1.02% ±1.34%
 buffers/buffer-tostring.js n=10000000 len=64 args=0 encoding='latin1'                                   ***      1.30 %       ±0.49% ±0.65% ±0.85%
 buffers/buffer-tostring.js n=10000000 len=64 args=0 encoding='utf8'                                     ***      2.31 %       ±0.98% ±1.31% ±1.72%
 buffers/buffer-tostring.js n=10000000 len=64 args=1 encoding='ascii'                                    ***      8.75 %       ±1.22% ±1.64% ±2.17%
 buffers/buffer-tostring.js n=10000000 len=64 args=1 encoding='hex'                                      ***      5.16 %       ±0.71% ±0.94% ±1.23%
 buffers/buffer-tostring.js n=10000000 len=64 args=1 encoding='latin1'                                   ***      7.52 %       ±1.41% ±1.88% ±2.46%
 buffers/buffer-tostring.js n=10000000 len=64 args=1 encoding='UCS-2'                                    ***      7.93 %       ±1.40% ±1.86% ±2.43%
 buffers/buffer-tostring.js n=10000000 len=64 args=1 encoding='utf8'                                     ***      9.29 %       ±1.19% ±1.60% ±2.11%
 buffers/buffer-tostring.js n=10000000 len=64 args=3 encoding='ascii'                                    ***      9.34 %       ±0.89% ±1.19% ±1.57%
 buffers/buffer-tostring.js n=10000000 len=64 args=3 encoding='hex'                                      ***      4.83 %       ±0.60% ±0.80% ±1.04%
 buffers/buffer-tostring.js n=10000000 len=64 args=3 encoding='latin1'                                   ***     10.68 %       ±1.50% ±2.01% ±2.66%
 buffers/buffer-tostring.js n=10000000 len=64 args=3 encoding='UCS-2'                                    ***      4.50 %       ±1.19% ±1.58% ±2.07%
 buffers/buffer-tostring.js n=10000000 len=64 args=3 encoding='utf8'                                     ***      9.86 %       ±0.84% ±1.12% ±1.48%

 buffers/buffer-write-string.js n=10000000 len=2048 args='' encoding=''                                   **      1.87 %       ±1.20% ±1.60% ±2.09%
 buffers/buffer-write-string.js n=10000000 len=2048 args='' encoding='ascii'                             ***      8.84 %       ±2.47% ±3.30% ±4.31%
 buffers/buffer-write-string.js n=10000000 len=2048 args='' encoding='hex'                               ***      1.24 %       ±0.56% ±0.75% ±0.98%
 buffers/buffer-write-string.js n=10000000 len=2048 args='' encoding='latin1'                            ***      8.56 %       ±1.02% ±1.36% ±1.77%
 buffers/buffer-write-string.js n=10000000 len=2048 args='' encoding='utf16le'                           ***      5.39 %       ±0.68% ±0.91% ±1.18%
 buffers/buffer-write-string.js n=10000000 len=2048 args='' encoding='utf8'                              ***      2.59 %       ±1.09% ±1.45% ±1.89%

 buffers/buffer-from.js n=4000000 len=100 source='string'                                                ***      3.36 %       ±1.69% ±2.24% ±2.92%

 buffers/buffer-indexof.js n=100000 type='buffer' encoding='ucs2' search='@'                             ***     13.54 %       ±4.29% ±5.71% ±7.43%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='ucs2' search='</i> to the Caterpillar'         *      0.57 %       ±0.48% ±0.63% ±0.82%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='ucs2' search='aaaaaaaaaaaaaaaaa'               *      0.44 %       ±0.43% ±0.58% ±0.75%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='ucs2' search='Alice'                         ***     18.63 %       ±4.71% ±6.27% ±8.16%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='ucs2' search='Gryphon'                       ***      3.35 %       ±1.91% ±2.55% ±3.31%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='ucs2' search='neighbouring pool'               *      0.48 %       ±0.46% ±0.61% ±0.80%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='ucs2' search='SQ'                            ***      3.65 %       ±1.69% ±2.25% ±2.92%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='utf8' search='@'                             ***     16.95 %       ±3.99% ±5.31% ±6.92%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='utf8' search='Alice'                         ***     18.55 %       ±5.48% ±7.30% ±9.52%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='utf8' search='Gryphon'                        **      3.03 %       ±1.78% ±2.36% ±3.08%
 buffers/buffer-indexof.js n=100000 type='buffer' encoding='utf8' search='Ou est ma chatte?'               *      0.78 %       ±0.66% ±0.88% ±1.14%
 buffers/buffer-indexof.js n=100000 type='string' encoding='ucs2' search='@'                             ***     14.38 %       ±5.38% ±7.16% ±9.32%
 buffers/buffer-indexof.js n=100000 type='string' encoding='ucs2' search='Alice'                         ***     27.09 %       ±5.71% ±7.60% ±9.91%
 buffers/buffer-indexof.js n=100000 type='string' encoding='ucs2' search='SQ'                            ***      3.63 %       ±2.07% ±2.75% ±3.58%
 buffers/buffer-indexof.js n=100000 type='string' encoding='utf8' search='@'                             ***     12.08 %       ±4.53% ±6.03% ±7.85%
 buffers/buffer-indexof.js n=100000 type='string' encoding='utf8' search='Alice'                         ***     15.36 %       ±5.71% ±7.60% ±9.90%
 buffers/buffer-indexof.js n=100000 type='string' encoding='utf8' search='Gryphon'                         *      1.88 %       ±1.87% ±2.48% ±3.23%
 buffers/buffer-indexof.js n=100000 type='string' encoding='utf8' search='Ou est ma chatte?'               *      0.88 %       ±0.78% ±1.04% ±1.36%
 buffers/buffer-indexof.js n=100000 type='string' encoding='utf8' search='SQ'                              *      3.60 %       ±3.04% ±4.04% ±5.27%
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

@mscdex mscdex changed the title Buffer consolidate encoding parsing buffer: consolidate encoding parsing Aug 20, 2019

@nodejs-github-bot

This comment has been minimized.

@Trott

This comment has been minimized.

Copy link
Member

commented Aug 22, 2019

@nodejs/buffer This could use some reviews.

@nodejs-github-bot

This comment has been minimized.

@bnoordhuis
Copy link
Member

left a comment

Nice work, some impressive speedups!

throw new ERR_INVALID_ARG_TYPE(
'string', ['string', 'Buffer', 'ArrayBuffer'], string
);
const encodingOps = {

This comment has been minimized.

Copy link
@bnoordhuis

bnoordhuis Aug 22, 2019

Member

Maybe add a comment that for performance reasons it's imperative that all the entries have the same shape / hidden map? (I.e., users of an ops object shouldn't go poly- or megamorphic.)

@Trott Trott added the author ready label Aug 23, 2019

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment has been minimized.

Copy link

commented Aug 23, 2019

src/node_buffer.cc Outdated Show resolved Hide resolved
src/node_buffer.cc Outdated Show resolved Hide resolved
src/node_buffer.cc Outdated Show resolved Hide resolved

@mscdex mscdex force-pushed the mscdex:buffer-consolidate-encoding-parsing branch from cb09531 to 9c7149a Aug 26, 2019

@nodejs-github-bot

This comment has been minimized.

mscdex added 2 commits Aug 20, 2019
buffer: consolidate encoding parsing
PR-URL: #29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
buffer: correct indexOf() error message
PR-URL: #29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>

@mscdex mscdex force-pushed the mscdex:buffer-consolidate-encoding-parsing branch from 9c7149a to 5dc5cfb Aug 28, 2019

@mscdex mscdex merged commit 5dc5cfb into nodejs:master Aug 28, 2019

1 check passed

Travis CI - Branch Build Passed
Details

@mscdex mscdex deleted the mscdex:buffer-consolidate-encoding-parsing branch Aug 28, 2019

BridgeAR added a commit that referenced this pull request Sep 3, 2019
buffer: consolidate encoding parsing
PR-URL: #29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
BridgeAR added a commit that referenced this pull request Sep 3, 2019
buffer: correct indexOf() error message
PR-URL: #29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@BridgeAR BridgeAR referenced this pull request Sep 3, 2019
BridgeAR added a commit that referenced this pull request Sep 4, 2019
buffer: consolidate encoding parsing
PR-URL: #29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
BridgeAR added a commit that referenced this pull request Sep 4, 2019
buffer: correct indexOf() error message
PR-URL: #29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
JeniaBR added a commit to JeniaBR/node that referenced this pull request Sep 11, 2019
buffer: consolidate encoding parsing
PR-URL: nodejs#29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
JeniaBR added a commit to JeniaBR/node that referenced this pull request Sep 11, 2019
buffer: correct indexOf() error message
PR-URL: nodejs#29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
JeniaBR added a commit to JeniaBR/node that referenced this pull request Sep 11, 2019
buffer: consolidate encoding parsing
PR-URL: nodejs#29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
JeniaBR added a commit to JeniaBR/node that referenced this pull request Sep 11, 2019
buffer: correct indexOf() error message
PR-URL: nodejs#29217
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.