Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BP-47 (task4): Aligned native buffer wrapper #3253

Merged

Conversation

hangc0276
Copy link
Contributor

Motivation

Task 4 of BP-47, aligned native buffer wrapper to support direct IO.

Changes

A utility buffer class to be used with JNA calls. Buffers are page aligned (4k pages).

The wrapper mostly handles writes between ByteBuffers and ByteBufs. It also provides a method for padding the buffer to the next alignment, so writes can have an aligned size also (as required by direct I/O). The padding is done with 0xF0, so that if it is read as an integer, or long, the value will be negative (assuming the read isa java read, and thus a signed int).

Master Issue: #2943 , Subtask of #2932

Others

The commit is made by @ivankelly . This is the sub task of #2932 , which pushed out by @mauricebarnum. However, this PR contains too many changes and the number of code lines reaches 8K+, and it is hard to review. According to your suggestion #2932 (comment), and after communicate with @mauricebarnum, we are planing to divide the PR into 6 PRs, Please refer to #2943 (comment).

However, @mauricebarnum doesn't have enough time to deal with those issues, and we desperately need this feature. After communicated with @mauricebarnum and @merlimat , I help to work on divide the PRs and push them out. We are hoping to contain this feature in BookKeeper 4.16.0

Ivan Kelly and others added 5 commits April 30, 2022 16:52
Utility to make it easier to add context to exception messages. Avoids
having to do custom formatting. You just add keyvalues to a builder.

Use like:
```
exMsg("something failed").kv("filename", fn).kv("errno", errno).toString()
```

(cherry picked from commit 1dd031d)
A utility buffer class to be used with JNA calls. Buffers are page
aligned (4k pages).

The wrapper mostly handles writes between ByteBuffers and
ByteBufs. It also provides a method for padding the buffer to the next
alignment, so writes can have an aligned size also (as required by
direct I/O). The padding is done with 0xF0, so that if it is read as
an integer, or long, the value will be negative (assuming the read is
a java read, and thus an signed int).

(cherry picked from commit 9c0e026)
@hangc0276
Copy link
Contributor Author

rerun failure checks

@hangc0276
Copy link
Contributor Author

rerun failure checks

@hangc0276
Copy link
Contributor Author

rerun failure checks

4 similar comments
@hangc0276
Copy link
Contributor Author

rerun failure checks

@hangc0276
Copy link
Contributor Author

rerun failure checks

@hangc0276
Copy link
Contributor Author

rerun failure checks

@hangc0276
Copy link
Contributor Author

rerun failure checks

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eolivelli eolivelli merged commit 8c661df into apache:master May 4, 2022
@hangc0276 hangc0276 self-assigned this Jul 25, 2022
@hangc0276 hangc0276 added this to the 4.16.0 milestone Jul 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants