8170305: URLConnection doesn't handle HTTP/1.1 1xx (informational) messages #10229

jaikiran · 2022-09-09T11:21:51Z

Can I please get a review of this change which proposes to fix https://bugs.openjdk.org/browse/JDK-8170305?

The commit in this PR changes the internal implementation of HttpURLConnection to ignore interim informational 1xx responses from server and continue to wait for the final response. This is a similar fix to what we did in the new HttpClient API in #10169.

A new test has been added to use the HttpURLConnection to reproduce the issue and verify the fix.

tier1, tier2 and tier3 testing is in progress with this change.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8170305: URLConnection doesn't handle HTTP/1.1 1xx (informational) messages

Reviewers

Daniel Fuchs (@dfuch - Reviewer)
Michael McMahon (@Michael-Mc-Mahon - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/10229/head:pull/10229
$ git checkout pull/10229

Update a local copy of the PR:
$ git checkout pull/10229
$ git pull https://git.openjdk.org/jdk pull/10229/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 10229

View PR using the GUI difftool:
$ git pr show -t 10229

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/10229.diff

…ssages

bridgekeeper · 2022-09-09T11:23:26Z

👋 Welcome back jpai! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2022-09-09T11:27:04Z

@jaikiran The following label will be automatically applied to this pull request:

net

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2022-09-09T11:31:19Z

Webrevs

dfuch

LGTM globally. We're not supposed to receive 101 if we didn't ask for an upgrade (and we never should), but what happens if we still receive 101?
It could be beneficial to add that to your test as well.

…onse too

…ttpURLConnection doesn't ask for a connection upgrade

jaikiran · 2022-09-12T07:37:32Z

Hello Daniel,
I've now updated the PR to include a test for 101 as well as updated the source to handle the 101 differently than other interim 1xx responses. Since HttpURLConnection doesn't support connection upgrade, the receipt of a 101 response from a server is considered an error and with the change in this PR, the request will now fail and we internally close the connection as well.

I've triggered tier1, tier2 and tier3 to make sure this doesn't cause any issues.

Michael-Mc-Mahon · 2022-09-12T08:34:23Z

I just tested the latest patch and the new test hangs for me. I can send you the stack trace to take a look.

jaikiran · 2022-09-12T09:40:00Z

Hello Michael, I could reproduce the hang locally after a few attempts. This was a bug in the test server which I've now fixed in the updated version of the PR.

What was happening was that the client upon seeing an unexpected 101 would close the connection and throw a (client side) IOException. The HttpURLConnection however has certain code which retries a (GET) request on IOException. The test server upon seeing a connection closure from the client would stop accept()ing any more connections but would still have an ESTABLISHED server socket listening on that host/port. So with no one "accepting" the connection, the client side would time out waiting for the server to respond. This has now been fixed in the test specific server and I've run this test a few times to make sure this now passes. I'll also run some CI runs with a --test-repeat just to be sure it doesn't have any other issues.

jaikiran · 2022-09-12T12:22:42Z

tier1, tier2, tier3 testing as well as testing of test/jdk/java/net/HttpURLConnection/ with a test-repeat of 50 completed without any issues, with the latest state of this PR.

dfuch · 2022-09-12T12:52:00Z

test/jdk/java/net/HttpURLConnection/Response1xxTest.java

+        System.out.println("Issuing request to " + requestURI);
+        final HttpURLConnection urlConnection = (HttpURLConnection) requestURI.toURL().openConnection();
+        // we expect the request to fail because the server unexpectedly sends a 101 response
+        Assert.assertThrows(IOException.class, () -> urlConnection.getResponseCode());


Should we explicitly wait for ProtocolException here, since it's what we're throwing?

I initially thought about that, but since the API javadoc doesn't specifically talk about this sub-type of IOException, I decided it's perhaps better not to start expecting it. However, if it's OK for the test to rely on this (internal) knowledge then I'll go ahead and change it.

Yes - this is not a conformance test, and ProtocolException is within the bounds. It's OK for a non-regression test to expect the specific subtype that the code change is throwing. My concern here is that the test could be passing if some other kind of errors occurred - like e.g. connect exception. Then we wouldn't be testing what we think we're testing.

Thank you. I've now updated the test to expect ProtocolException. The test continues to pass with this change.

openjdk · 2022-09-12T13:34:48Z

@jaikiran This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8170305: URLConnection doesn't handle HTTP/1.1 1xx (informational) messages

Reviewed-by: dfuchs, michaelm

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 17 new commits pushed to the master branch:

9ef6c09: 8287908: Use non-cloning reflection methods where acceptable
0c61bf1: 8293282: LoadLibraryUnloadTest.java fails with "Too few cleared WeakReferences"
91c9091: 8293343: sun/management/jmxremote/bootstrap/RmiSslNoKeyStoreTest.java failed with "Agent communication error: java.io.EOFException"
4c77bd3: 6529151: NullPointerException in swing.plaf.synth.SynthLookAndFeel$Handler
d5aae01: 8293544: G1: Add comment in G1BarrierSetC1::pre_barrier
37df5f5: 8291599: Assertion in PhaseIdealLoop::skeleton_predicate_has_opaque after JDK-8289127
699c429: 8292866: Java_sun_awt_shell_Win32ShellFolder2_getLinkLocation check MultiByteToWideChar return value for failures
68da02c: 8292240: CarrierThread.blocking not reset when spare not activated
005b49b: 8293044: C1: Missing access check on non-accessible class
91d00b3: 8288473: Remove unused frame::set_pc_preserve_deopt methods
... and 7 more: https://git.openjdk.org/jdk/compare/9d6b0285f53599816c30393b87d16772ef6314b7...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

Michael-Mc-Mahon

LGTM

jaikiran · 2022-09-13T05:06:07Z

Thank you Daniel and Michael for the reviews.

jaikiran · 2022-09-13T05:06:12Z

/integrate

openjdk · 2022-09-13T05:08:19Z

Going to push as commit 8bd79d3.
Since your change was applied there have been 27 commits pushed to the master branch:

9cd3e35: 4834298: JFileChooser.getSelectedFiles() failed with multi-selection and double-click
ec2629c: 8275275: AArch64: Fix performance regression after auto-vectorization on NEON
cbee0bc: 8292587: AArch64: Support SVE fabd instruction
68645eb: 8293566: RISC-V: Clean up push and pop registers
526eb54: 8293669: SA: Remove unnecssary "InstanceStackChunkKlass: InstanceStackChunkKlass" output when scanning heap
41ce658: 8292225: Rename ArchiveBuilder APIs related to source and buffered addresses
155b10a: 8293329: x86: Improve handling of constants in AES/GHASH stubs
d3f7e3b: 8293339: vm/jvmti/StopThread/stop001/stop00103 crashes with SIGSEGV in Continuation::is_continuation_mounted
524af94: 8283627: Outdated comment in MachineDescriptionTwosComplement.isLP64
cea409c: 8292738: JInternalFrame backgroundShadowBorder & foregroundShadowBorder line is longer in Mac Look and Feel
... and 17 more: https://git.openjdk.org/jdk/compare/9d6b0285f53599816c30393b87d16772ef6314b7...master

Your commit was automatically rebased without conflicts.

openjdk · 2022-09-13T05:08:32Z

@jaikiran Pushed as commit 8bd79d3.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

reschke · 2022-09-13T17:37:11Z

Thanks - I'll test once the next build is out.

Is this something that would be a candidate for backporting?

jaikiran · 2022-09-14T00:54:48Z

Hello Julian,

Is this something that would be a candidate for backporting?

This change is relatively straightforward and backporting should be OK. OpenJDK backports however are decided/managed by different team(s) and discussed in https://mail.openjdk.org/mailman/listinfo/jdk-updates-dev mailing list. The OpenJDK guide has more details on the backporting process https://openjdk.org/guide/#backporting

reschke · 2022-09-17T10:53:31Z

Tested over at https://github.com/greenbytes/java-http-1xx-tests and found to be working; thanks!

OpenJDK backports however are decided/managed by different team(s) and discussed in https://mail.openjdk.org/mailman/listinfo/jdk-updates-dev mailing list.

So is the proposal that I should follow up over there?

jaikiran · 2022-09-18T05:31:22Z

Tested over at https://github.com/greenbytes/java-http-1xx-tests and found to be working; thanks!

Thank you Julian for running those tests. Glad to hear it works now.

OpenJDK backports however are decided/managed by different team(s) and discussed in https://mail.openjdk.org/mailman/listinfo/jdk-updates-dev mailing list.

So is the proposal that I should follow up over there?

Yes, I suggest asking in that mailing list on how to get this backported. I would have given you a bit more precise instructions, but the last time I backported a change was way before the new bots based backporting was introduced, so I don't have more recent knowledge of backports.

8170305: URLConnection doesn't handle HTTP/1.1 1xx (informational) me…

3576e61

…ssages

openjdk bot added the rfr Pull request is ready for review label Sep 9, 2022

openjdk bot added the net net-dev@openjdk.org label Sep 9, 2022

dfuch reviewed Sep 9, 2022

View reviewed changes

101 Upgrade isn't supported by HttpURLConnection, so ignore that resp…

bd887e9

…onse too

jaikiran mentioned this pull request Sep 9, 2022

8292044: HttpClient doesn't handle 102 or 103 properly #10169

Closed

3 tasks

consider it a protocol violation if the server sends a 101 when the H…

0ede368

…ttpURLConnection doesn't ask for a connection upgrade

Add a log message when ignorning 1xx responses

08e681b

fix test server to avoid hang in test

c4b62c5

dfuch reviewed Sep 12, 2022

View reviewed changes

verify ProtocolException in the test

cb9364a

dfuch approved these changes Sep 12, 2022

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Sep 12, 2022

Michael-Mc-Mahon approved these changes Sep 12, 2022

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Sep 13, 2022

openjdk bot closed this Sep 13, 2022

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 13, 2022

jaikiran deleted the 8170305 branch September 13, 2022 05:09

8170305: URLConnection doesn't handle HTTP/1.1 1xx (informational) messages #10229

8170305: URLConnection doesn't handle HTTP/1.1 1xx (informational) messages #10229

Uh oh!

Conversation

jaikiran commented Sep 9, 2022 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Sep 9, 2022

Uh oh!

openjdk bot commented Sep 9, 2022

Uh oh!

mlbridge bot commented Sep 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

dfuch left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaikiran commented Sep 12, 2022

Uh oh!

Michael-Mc-Mahon commented Sep 12, 2022

Uh oh!

jaikiran commented Sep 12, 2022

Uh oh!

jaikiran commented Sep 12, 2022

Uh oh!

dfuch Sep 12, 2022

Choose a reason for hiding this comment

Uh oh!

jaikiran Sep 12, 2022

Choose a reason for hiding this comment

Uh oh!

dfuch Sep 12, 2022

Choose a reason for hiding this comment

Uh oh!

jaikiran Sep 12, 2022

Choose a reason for hiding this comment

Uh oh!

openjdk bot commented Sep 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Michael-Mc-Mahon left a comment

Choose a reason for hiding this comment

Uh oh!

jaikiran commented Sep 13, 2022

Uh oh!

jaikiran commented Sep 13, 2022

Uh oh!

openjdk bot commented Sep 13, 2022

Uh oh!

openjdk bot commented Sep 13, 2022

Uh oh!

reschke commented Sep 13, 2022

Uh oh!

jaikiran commented Sep 14, 2022

Uh oh!

reschke commented Sep 17, 2022

Uh oh!

jaikiran commented Sep 18, 2022

Uh oh!

Uh oh!

jaikiran commented Sep 9, 2022 •

edited by openjdk bot

Loading

mlbridge bot commented Sep 9, 2022 •

edited

Loading

dfuch left a comment •

edited

Loading

openjdk bot commented Sep 12, 2022 •

edited

Loading