Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash: Create ByteBuffer Slice from ByteBuffer Slice after 16MB #1813

Merged
merged 3 commits into from Apr 22, 2021

Conversation

fabianfett
Copy link
Member

@fabianfett fabianfett commented Apr 21, 2021

Fixes an issue that could lead to an assert in Debug mode and to undefined behavior in Release mode.

Motivation:

  • When reading a new ByteBuffer slice from an originating ByteBuffer slice where the new ByteBuffer slice starts more than 16MiB into the underlying ByteBuffer storage, the new ByteBuffer slice will get a new 0 based storage.
  • When creating the storage for the new slice we have an issue in where storage based indices were used which should have been slice based indices.

Modifications:

  • Using the correct indices, when creating a ByteBuffer slice which is more than 16MiB into its originating storage.

Result:

@Lukasa Lukasa added the patch-version-bump-only For PRs that when merged will only cause a bump of the patch version, ie. 1.0.x -> 1.0.(x+1) label Apr 21, 2021
Copy link
Contributor

@Lukasa Lukasa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this patch is great, it does a bit too much reaching into the ByteBuffer internals for my comfort. Can we rewrite it to avoid using UInt24 and avoid touching the _slice directly?

Copy link
Member

@weissi weissi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much @fabianfett , this is really fantastic work!!

// 16MiB after the originating backing storage.

// create a buffer with (16MiB - 1byte) + 2 bytes = 16MiB + 1 byte
let inputBufferLength = Int(_UInt24.max) + 2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just hardcode the value here. The _ means that _UInt24 is meant to be private. So let inputBufferLength = 1 << 24 + 1 or similar.

Tests/NIOTests/ByteBufferTest.swift Outdated Show resolved Hide resolved
Tests/NIOTests/ByteBufferTest.swift Outdated Show resolved Hide resolved
Tests/NIOTests/ByteBufferTest.swift Outdated Show resolved Hide resolved
let finalSlice = remainingSlice.readSlice(length: finalSliceLength)
XCTAssertEqual(finalSlice?.storageCapacity, 1)
XCTAssertEqual(finalSlice?._slice.lowerBound, 0)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should additionally assert

  • that the slices/values read have the correct content (1s, 2s, and 3s or so as suggested above)
  • that the underlying storage has indeed changed. There's a (test only) property called buffer.storageIdentifier or so which stays the same if nothing got reallocated/copied and changes if it has

Copy link
Contributor

@Lukasa Lukasa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor cleanup and then I think this is good.

Tests/NIOTests/ByteBufferTest.swift Outdated Show resolved Hide resolved
Co-authored-by: Cory Benfield <lukasa@apple.com>
@fabianfett fabianfett requested a review from Lukasa April 22, 2021 07:59
Copy link
Contributor

@Lukasa Lukasa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice patch!

Copy link
Member

@weissi weissi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, I left one suggestion which you may want to apply. We prefer lines <= 120 (IIRC) and Array<UInt8>(...) over [UInt8](...) for the constructors.

Tests/NIOTests/ByteBufferTest.swift Outdated Show resolved Hide resolved
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
patch-version-bump-only For PRs that when merged will only cause a bump of the patch version, ie. 1.0.x -> 1.0.(x+1)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants