Implement #8057 103 Early Hint #8058

gregw · 2022-05-26T00:03:38Z

Implementing #8057 Early hints

gregw · 2022-05-26T03:29:39Z

@lorban can you look at why h3 is failing.

lorban · 2022-05-26T06:34:52Z

@gregw I see no H3 test failure in the PR's build, only a checkstyle error. What am I overlooking?

gregw · 2022-05-26T06:44:01Z

@lorban have you run org.eclipse.jetty.http.client.IntermediateResponseTest#test103EarlyHint ?

lorban · 2022-05-26T14:22:13Z

@gregw I've added the missing support in H3 for 103 status code and pushed the changes to this branch.

While your test now passes and I'm reasonably confident my changes are correct, it would still be wise to have @sbordet review this bit eventually.

jetty-http/src/main/java/org/eclipse/jetty/http/HttpGenerator.java

...-http2/http2-server/src/main/java/org/eclipse/jetty/http2/server/HttpTransportOverHTTP2.java

...p-client-transport/src/test/java/org/eclipse/jetty/http/client/IntermediateResponseTest.java

...tp3-server/src/main/java/org/eclipse/jetty/http3/server/internal/HttpTransportOverHTTP3.java

jetty-server/src/main/java/org/eclipse/jetty/server/Response.java

...p-client-transport/src/test/java/org/eclipse/jetty/http/client/IntermediateResponseTest.java

gregw · 2022-05-26T23:50:58Z

@lorban @sbordet I've updated for review feedback.
However, there is still something racy either in the tests or the impl. UNIX_DOMAIN frequently fails for me if run with other tests (the second 103 is skipped). I have once seen H2 fail by the 103 being reported as the final response instead of the 200, but I can't reproduce. Something is not quiet right :(

gregw · 2022-05-26T23:52:11Z

Hmmm and I've just had a flake of the 102 test for UNIX_DOMAIN as well... also not reproducible :(

gregw · 2022-05-26T23:58:26Z

@sbordet what does the client do if it is still processing the 102/103 response and the final (or next) response turns up? On the server, we don't wait for anything from the client before sending the next response, but if I put in a 10ms sleep between the responses then I don't get any flakes. Either we are writing over the responses or the client is processing the next one before the last is finished?

gregw · 2022-05-27T00:01:38Z

@sbordet I think the race is in the client. If I put the 10ms sleeps in the server I pass every time. However, if I put a 100ms sleep in the onSuccess handling for the 1xx responses, then I fail more often than not, even for HTTP.

lorban · 2022-05-27T07:10:32Z

jetty-server/src/main/java/org/eclipse/jetty/server/Response.java

+     * Send a 103 response as per <a href="https://datatracker.ietf.org/doc/html/rfc8297">RFC8297</a>
+     * This method is called by sendError if it is passed 103.
+     *
+     * @throws IOException if unable to send the 102 response


- if unable to send the 102 response + if unable to send the 103 response

lorban · 2022-05-27T07:13:10Z

All tests in IntermediateResponseTest eventually fail when ran enough times, even if the H3 and UNIX_DOMAIN ones are more sensible.

Added test harness for intermediary responses.

Implemented 103 early hint, but is failing for HTTP>=2

Fixed H2 Attempted fix H3

fixed checkstyle

Signed-off-by: Ludovic Orban <lorban@bitronix.be>

updates from review

gregw · 2022-05-30T01:15:14Z

@sbordet @lorban The flaky issue is somewhere below the generation of the response in the server and the parsing of it in the client. I added some println's and for a good run I see:

generateResponseLine HTTP/1.1{s=103,h=2,cl=-1}
startResponse 103 Early Hint
generateResponseLine HTTP/1.1{s=103,h=2,cl=-1}
startResponse 103 Early Hint
generateResponseLine HTTP/1.1{s=200,h=2,cl=-1}
startResponse 200 OK

So each generated response is parsed (startResponse) in the client.

But for a bad run I see:

generateResponseLine HTTP/1.1{s=103,h=2,cl=-1}
generateResponseLine HTTP/1.1{s=103,h=2,cl=-1}
startResponse 103 Early Hint
generateResponseLine HTTP/1.1{s=200,h=2,cl=-1}
startResponse 200 OK

So all the responses are generated, but one of them is not received. It feels like something might be clearing a buffer when it should not?

@lorban can you focus on the H2/H3 flakes and I'll keep looking at h1

updates from review

gregw · 2022-05-30T01:27:44Z

oops sorry forced pushed after a rebase

Found (but not fixed) flake.

gregw · 2022-05-30T02:18:30Z

jetty-client/src/main/java/org/eclipse/jetty/client/http/HttpReceiverOverHTTP.java

+                // TODO this can clean a pipelined response after a 1xx
                if (LOG.isDebugEnabled())
                    LOG.debug("Discarding unexpected content after response: {}", networkBuffer);
                networkBuffer.clear();


@sbordet @lorban this is the cause of the UNIX_DOMAIN flake. A pipelined response can be discarded after a 1xx response. I'm not sure of the fix.

@sbordet

Fixed flake. @sbordet please review this commit!

gregw · 2022-05-30T03:15:42Z

@sbordet I found and maybe fixed the flake in the HttpClient (discarding content after a 1xx), but would appreciate very careful review.
@lorban I can't get any flaky fails in h2 or h3? How about you?

lorban · 2022-05-30T10:05:37Z

@gregw I can make HttpClientStreamTest.testFileUpload on H3 fail about once every couple dozen runs (I'm aware of that failure and I already started investigating it) but I cannot manage to get any of the tests in InformationalResponseTest to fail for H2 or H3 anymore.

That's strange...

sbordet · 2022-05-30T10:32:53Z

@gregw looking

Improved implementation on the client-side. Introduced HttpStatus.isInterim() for 1XX codes that are not 101. Signed-off-by: Simone Bordet <simone.bordet@gmail.com>

gregw self-assigned this May 26, 2022

gregw linked an issue May 26, 2022 that may be closed by this pull request

Support Http Response 103 (Early Hints) #8057

Closed

gregw added this to In progress in Jetty 10.0.10/11.0.10 - 🧊 FROZEN 🥶 via automation May 26, 2022

sbordet requested changes May 26, 2022

View reviewed changes

Jetty 10.0.10/11.0.10 - 🧊 FROZEN 🥶 automation moved this from In progress to Review in progress May 26, 2022

lorban reviewed May 27, 2022

View reviewed changes

gregw requested a review from sbordet May 28, 2022 06:09

gregw and others added 7 commits May 30, 2022 10:35

Implement #8057 103 Early Hint

8b06eb0

Added test harness for intermediary responses.

Implement #8057 103 Early Hint

a3bbbc0

Implemented 103 early hint, but is failing for HTTP>=2

Implement #8057 103 Early Hint

52a6894

Fixed H2 Attempted fix H3

Implement #8057 103 Early Hint

29741a2

Fixed H2 Attempted fix H3

Implement #8057 103 Early Hint

12a0049

fixed checkstyle

add missing H3 support for status 103

8470f49

Signed-off-by: Ludovic Orban <lorban@bitronix.be>

Implement #8057 103 Early Hint

41532ed

updates from review

Implement #8057 103 Early Hint

9d40cd4

updates from review

gregw force-pushed the jetty-10.0.x-8057-103EarlyHint branch from bfa8c65 to 9d40cd4 Compare May 30, 2022 01:27

Implement #8057 103 Early Hint

9451723

Found (but not fixed) flake.

gregw commented May 30, 2022

View reviewed changes

Implement #8057 103 Early Hint

01bf0f4

Fixed flake. @sbordet please review this commit!

Issue #8057 - Support Http Response 103 (Early Hints)

d0cfd7c

Improved implementation on the client-side. Introduced HttpStatus.isInterim() for 1XX codes that are not 101. Signed-off-by: Simone Bordet <simone.bordet@gmail.com>

gregw marked this pull request as ready for review May 31, 2022 00:08

gregw requested a review from lorban May 31, 2022 00:08

lorban approved these changes May 31, 2022

View reviewed changes

sbordet approved these changes May 31, 2022

View reviewed changes

Jetty 10.0.10/11.0.10 - 🧊 FROZEN 🥶 automation moved this from Review in progress to Reviewer approved May 31, 2022

gregw merged commit 7a1c165 into jetty-10.0.x Jun 1, 2022

Jetty 10.0.10/11.0.10 - 🧊 FROZEN 🥶 automation moved this from Reviewer approved to Done Jun 1, 2022

gregw deleted the jetty-10.0.x-8057-103EarlyHint branch June 1, 2022 01:57

gregw added the Sponsored This issue affects a user with a commercial support agreement label Jun 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement #8057 103 Early Hint #8058

Implement #8057 103 Early Hint #8058

gregw commented May 26, 2022

gregw commented May 26, 2022

lorban commented May 26, 2022

gregw commented May 26, 2022

lorban commented May 26, 2022 •

edited

gregw commented May 26, 2022

gregw commented May 26, 2022

gregw commented May 26, 2022

gregw commented May 27, 2022

lorban May 27, 2022

lorban commented May 27, 2022

gregw commented May 30, 2022

gregw commented May 30, 2022

gregw May 30, 2022

gregw commented May 30, 2022

lorban commented May 30, 2022 •

edited

sbordet commented May 30, 2022

Implement #8057 103 Early Hint #8058

Implement #8057 103 Early Hint #8058

Conversation

gregw commented May 26, 2022

gregw commented May 26, 2022

lorban commented May 26, 2022

gregw commented May 26, 2022

lorban commented May 26, 2022 • edited

gregw commented May 26, 2022

gregw commented May 26, 2022

gregw commented May 26, 2022

gregw commented May 27, 2022

lorban May 27, 2022

Choose a reason for hiding this comment

lorban commented May 27, 2022

gregw commented May 30, 2022

gregw commented May 30, 2022

gregw May 30, 2022

Choose a reason for hiding this comment

gregw commented May 30, 2022

lorban commented May 30, 2022 • edited

sbordet commented May 30, 2022

lorban commented May 26, 2022 •

edited

lorban commented May 30, 2022 •

edited