Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileRegion support in HTTP/2 #3928

Closed
wants to merge 1 commit into from
Closed

Conversation

Apache9
Copy link
Contributor

@Apache9 Apache9 commented Jul 3, 2015

POC of #3927.

@netkins
Copy link

netkins commented Jul 3, 2015

AUTOMATED MESSAGE for ADMINS:
Please verify and accept the pull request.
To accept, say @netkins accept
To build again, say @netkins build
To whitelist the author, say @netkins whitelist

/**
* Wrap a DATA frame so it can be written subject to flow-control. Note that this implementation assumes it
* only writes padding once for the entire payload as opposed to writing it once per-frame. This makes the
* {@link #size} calculation deterministic thereby greatly simplifying the implementation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{@link #size} -> {@link #size()}

@Scottmitch Scottmitch added this to the 4.1.0.Beta6 milestone Jul 3, 2015
@Scottmitch Scottmitch self-assigned this Jul 3, 2015
@Scottmitch
Copy link
Member

@netkins accept

@Scottmitch
Copy link
Member

@Apache9 - I took a first pass. Looks good so far! Ping me with questions and when ready for another round of review.

@Apache9
Copy link
Contributor Author

Apache9 commented Jul 4, 2015

@Scottmitch Great! Let me address the issues you mentioned. Thanks.

@Apache9
Copy link
Contributor Author

Apache9 commented Jul 6, 2015

@Scottmitch I have updated the patch. Thanks.

@@ -400,10 +417,10 @@ public void operationComplete(ChannelFuture future) throws Exception {

@Override
public boolean merge(Http2RemoteFlowController.FlowControlled next) {
if (FlowControlledData.class != next.getClass()) {
if (FlowControlledByteBuf.class != next.getClass()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just getClass() != next.getClass()?

@nmittler
Copy link
Member

nmittler commented Jul 6, 2015

@Apache9 took a first pass ... just a few comments. Looks pretty good overall!

@Override
public void write(int allowedBytes) {
int bytesWritten = 0;
if (data == null || (allowedBytes == 0 && size != 0)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @nmittler's above comment. Either try to unify the 2 algorithms or keep them separate.

It may be good to unify these two methods to make sure both cases work. For example in this case you are setting data = null to signify you are done writing data, and may have padding left...but here you are checking data == null and exiting before checking if any padding needs to be written. The ByteBuf accounts for this by only setting data = null on release, and EMPTY_BUF if the write process exhausted the available data in the buffer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

o, it is ||, not &&. Let me fix it.
I separated the logic of writing padding only and writing frame because we do not have an EmptyFileRegion and it makes no sense to have an EmptyFileRegion I think? There must be a file associate with a FileRegion.

@Scottmitch
Copy link
Member

@Apache9 - Few more comments. Lets also add a unit test where the size is bigger than Integer.MAX_VALUE.

@Apache9
Copy link
Contributor Author

Apache9 commented Jul 6, 2015

Lets also add a unit test where the size is bigger than Integer.MAX_VALUE.

Yes, let me try.
And for FileRegion, FileRegionHelper is only used to prevent duplicate codes. Let me try to find another way. Thanks.

Add FileRegion support in HTTP/2

Modifications:

Add a writeData method in Http2DataWriter which accept FileRegion as data.
Add a readSlice method in FileRegion which is like the readSlice method in ByteBuf.
Change pendingBytes in DefaultHttp2RemoteFlowController and the return value of FlowControlled.size() to long.
Add two testcases to test the functionality of FileRegion support in HTTP/2.

Result:

Now one can write a FileRegion as DATA frame.
@Apache9
Copy link
Contributor Author

Apache9 commented Jul 9, 2015

@Scottmitch I have updated the patch. Rewrite DefaultFileRegion that add readSlice and unwrap. Add a canWriteFileRegionLargerThan2GB testcase in DefaultHttp2ConnectionEncoderTest.

Thanks.

@nmittler
Copy link
Member

nmittler commented Jul 9, 2015

@Scottmitch @normanmaurer I wonder if there might be a way to provide a common interface between FileRegion and ByteBuf to more easily support cases like this. WDYT?

Apache9 referenced this pull request in nmittler/netty Jul 11, 2015
Motivation:

Based on netty#3965. As a first step of integrating the unified reading API, the low-hanging fruit is to refactor `FIleRegion` to use it.

Modifications:

Refactored `FileRegion` to extend `ReadableObject` and implemented the new interface in `DefaultFileRegion`.

Result:

`FileRegion` implements the new `ReadableObject` interface.
@normanmaurer normanmaurer modified the milestones: 4.1.0.Beta6, 4.1.0.RC1 Sep 2, 2015
@Scottmitch Scottmitch assigned nmittler and unassigned Scottmitch Sep 16, 2015
@Scottmitch
Copy link
Member

@nmittler - Reassigned to you because of your related PR #3969

@normanmaurer normanmaurer modified the milestones: 4.1.0.CR1, 4.1.0.Final Nov 27, 2015
@normanmaurer normanmaurer modified the milestones: 4.1.0.Final, 4.1.0.CR7 Apr 9, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants