Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Buffer.slice unexpected behavior #9096

Closed
rideg opened this issue Oct 14, 2016 · 1 comment
Closed

Buffer.slice unexpected behavior #9096

rideg opened this issue Oct 14, 2016 · 1 comment
Labels
buffer Issues and PRs related to the buffer subsystem.

Comments

@rideg
Copy link

rideg commented Oct 14, 2016 •
edited by thefourtheye
Loading

  • Version: v6.7.0
  • Platform: Linux 3.13.0-96-generic Ubuntu SMP x86_64 GNU/Linux
  • Subsystem: NodeBuffer

It seems if one provides a non integer boundary to the slice function then the later slice calls on the same buffer provide wrong chunks.

Test case to reproduce

it("Should slice Buffer", () => {
    let original = new Buffer("1234567890ABCDEFG", "ascii");
    let chunk1 = original.slice(0, original.length / 3);
    let chunk2 = original.slice(original.length / 3);
    let result = Buffer.concat([chunk2, chunk1]);
    expect(result).to.be.eqls(original);
});

And the result is:

AssertionError: expected 
|Buffer 31 32 33 34 35 36 37 38 39 30 41 42 43 44 45 46| to deeply equal |Buffer 31 32 33 34 35 36 37 38 39 30 41 42 43 44 45 46 `**47**`|

In this test case the result buffer misses the last byte from the original one.

I am not exactly sure what should be the correct behavior but I would be fine either with throwing an exception or round/ceil/floor the provided boundary value.
@vkurchatkin vkurchatkin added the buffer Issues and PRs related to the buffer subsystem. label Oct 14, 2016
@mscdex
Copy link
Contributor

mscdex commented Oct 14, 2016

/cc @nodejs/buffer

@thefourtheye thefourtheye mentioned this issue Oct 14, 2016
Closed
3 tasks
thefourtheye added a commit to thefourtheye/io.js that referenced this issue Oct 15, 2016
@thefourtheye
buffer: coerce slice parameters consistently
9fc562c 
As shown in nodejs#9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: nodejs#9096
jasnell pushed a commit that referenced this issue Oct 18, 2016
@thefourtheye @jasnell
buffer: coerce slice parameters consistently
01c7796 
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
jasnell pushed a commit that referenced this issue Oct 19, 2016
@thefourtheye @jasnell
buffer: coerce slice parameters consistently
14d1a8a 
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
@cjihrig cjihrig mentioned this issue Oct 28, 2016
Merged
3 tasks
cjihrig added a commit to cjihrig/node that referenced this issue Nov 5, 2016
@cjihrig
buffer: coerce offset using Math.trunc()
a02b13f 
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: nodejs#9096
Refs: nodejs#9101
PR-URL: nodejs#9341
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
evanlucas pushed a commit that referenced this issue Nov 7, 2016
@cjihrig @evanlucas
buffer: coerce offset using Math.trunc()
c70c96a 
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: #9096
Refs: #9101
PR-URL: #9341
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
MylesBorins pushed a commit that referenced this issue Nov 18, 2016
@thefourtheye
buffer: coerce slice parameters consistently
8195b17 
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
MylesBorins pushed a commit that referenced this issue Nov 18, 2016
@cjihrig
buffer: coerce offset using Math.trunc()
ee0c28b 
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: #9096
Refs: #9101
PR-URL: #9341
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
MylesBorins pushed a commit that referenced this issue Nov 19, 2016
@thefourtheye @MylesBorins
buffer: coerce slice parameters consistently
0276e9e 
As shown in #9096, the offset and
end value of the `slice` call are coerced to numbers and then passed to
`FastBuffer`, which internally truncates the mantissa part if the number
is actually a floating point number. This actually affects the new
length of the slice calculation. For example,

    > const original = Buffer.from('abcd');
    undefined
    > original.slice(original.length / 3).toString()
    'bc'

This happens because, starting value of the slice is 4 / 3, which is
1.33 (approximately). Now, the length of the slice is calculated as
the difference between the actual length of the buffer and the starting
offset. So, it becomes 2.67 (4 - 1.33). Now, a new `FastBuffer` is
constructed, with the following values as parameters,

 1. actual buffer object,
 2. starting value, which is 1.33 and
 3. the length 2.67.

The underlying C++ code truncates the numbers and they become 1 and 2.
That is why the result is just `bc`.

This patch makes sure that all the offsets are coerced to integers
before any calculations are done.

Fixes: #9096
PR-URL: #9101
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
MylesBorins pushed a commit that referenced this issue Nov 19, 2016
@cjihrig @MylesBorins
buffer: coerce offset using Math.trunc()
0cffa3c 
This is a partial revert of
14d1a8a, which coerced the offset
of Buffer#slice() using the | operator. This causes some edge
cases to be handled incorrectly. This commit restores the old
behavior, but converts offsets to integers using Math.trunc().
This commit does not revert any tests, and adds an additional
regression test.

Refs: #9096
Refs: #9101
PR-URL: #9341
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
buffer Issues and PRs related to the buffer subsystem.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mscdex @vkurchatkin @rideg