Undo JDK transport demote

[cstamas]

New feature, improvement proposal

Undo #1599

[kwin]

Can you somewhere outline the reasons for this back and forth on Java HTTP client?

[cstamas]

There is a thread on Maven slack outlining it, check there please.

[kwin]

For the record: https://the-asf.slack.com/archives/C7Q9JB404/p1759231362859829

Volkan Yazıcı
Tuesday at 1:22 PM
AFAIU, Maven 4, which will baseline Java 17, will continue using Apache HTTP Client. If so, curious: why not the HttpClient shipped in Java 11?
96 replies

Konrad Windszus
Tuesday at 1:37 PM
This is an old/stale branch, please check out master which has https://github.com/apache/maven-resolver/blob/master/maven-resolver-transport-jdk-[…]ava/org/eclipse/aether/transport/jdk/JdkTransporterFactory.java
1:37
However it is not the default
1:38
for a reasoning please see 2ca95ee
1:38
Maybe @cstamas can share more details

cstamas
Tuesday at 1:40 PM
all maven 4 versions since some alpha has j.n.h.HttpClient transport
1:40
since Resolver 2.x is used in mvn4
1:40
but
1:40
as time passed with got a ton of feedback from users about it, and we "demoted it" step by step: for start we had to disable (by def) HTTP/2 as it proved "always break" in some environments
1:41
so latest RC used j.n.h.HttpClient as "simple" HTTP/1.1 client (like the old apache transport), but alas was not "on par", as it does not support some proxy setups apache does
1:42
and finally, upcoming Resolver 2.0.12 we totally demoted it, is not even default transport anymore, due new SSL/TLS issues has
1:42
but user can still opt to use it
1:42
https://github.com/apache/maven/blob/master/api/maven-api-core/src/main/java/org/apache/maven/api/Constants.java#L432

Volkan Yazıcı
Tuesday at 1:44 PM
@Konrad Windszus @cstamas Would you be okay with if I share this conversation with some OpenJDK members?

cstamas
Tuesday at 1:44 PM
sure
1:44
there are even JDK jiras for these
1:44
so they should be aware of it

Volkan Yazıcı
Tuesday at 1:45 PM
@cstamas Would you mind pointing me to the
"new SSL/TLS issues",
"it proved 'always break' in some environments", and
"JDK jiras", please?

cstamas
Tuesday at 1:45 PM
https://issues.apache.org/jira/browse/MNG-8145
1:45
the well known GOAWAY issue
1:45
cannot be handled
1:46
in these evnironments it is INEVITABLE to get GOAWAY, but httpClient does not expose ANY MEANS to handle it
1:46
(aside of some exception message contains assertion or alike)
1:51
fixed only in latest Javas https://bugs.openjdk.org/browse/JDK-8335181 for example, and this assume that all maven users update JDKs to latest, which is not always the case, and we can get flooded with issues we cannot handle, etc
1:55
btw, I agree that having new Maven 4 with ancient (default) transport using ASF HttpClient 4.x is far, very far from ideal, but the Java HttpClient proved too limited, while for other transport like new ASF HttpClient 5.x transport it seems there is no interest #742 nor in maven project nor in httpcomponents project (edited)
1:56
I alone cannot get to it, maybe later

cstamas
Tuesday at 2:03 PM
oh, also issues like these https://bugs.openjdk.org/browse/JDK-8283544 prevents transport to use services like jitpack jitpack/jitpack.io#7186
2:03
so many little annoying but not solvable issues

cstamas
Tuesday at 2:12 PM
not solvable == as it is not a library but part of JDK 🙂 so we cannot just "up the version" and release, but we need to talk users into upgrading their JDK
2:12
and that is many times not so simple

cstamas
Tuesday at 2:19 PM
TBH, for simple cases (like for example for me, simple home network, nothing fancy, going directly for Central) JDK HttpClient w/ HTTP/2 was working very well and was fast, on the builds I usually build, but on larger build, like of those of quarkus or camel, it knew to "suddenly die" and break the build
2:21
so I think JDK HttpClient may be considered as Wagon HTTP Lightweight was, good to "bootstrap" things, but....
2:22
@Romain ^ something to add? He was one of the users yelling when I "demoted" it, he also says "works for him" but at the cost of some "tuning" (JDK props)
2:25
@Guillaume Nodet ^

Romain
Tuesday at 3:44 PM
well for me it always worked without tuning, the tuning mention was using the JRE HttpClient in some apps not maven - but guess it converges at some point 😉
but I got way more issues with apache http client lately
while the transport uses dep mechanism and can be set globally in the settings or ~/.mavenrc I guess it is ok 👼

Volkan Yazıcı
Tuesday at 8:25 PM
Using the cited issues, I've compiled the following:
Incorrect handling of GOAWAY – MNG-8145 JDK-8335181 (delivered in Java 24, ported to 17 & 21)
GET method adds Content-Length: 0 header, rendering services like JitPack unusable – jitpack/jitpack.io#7186 JDK-8283544 (delivered in Java 19)
I see JDK-8283544 is marked as a 11 & 17 backport candidate, though not processed yet.
If we would also get JDK-8283544 into 17u, would you consider introducing JDK's HttpClient as the default?

cstamas
Tuesday at 8:26 PM
Yes, I'd love to (and enable HTTP/2 on it), as I said, it is very good in these scenarios
8:27
there is a third issue, something regarding TLS/SSL, but to be frank, I just "heard" about that issue (and our CI was failing due SSL/TLS handshake) and it was just another reason added, but maybe not recorded in form of issue/JIRA

Volkan Yazıcı
Tuesday at 8:41 PM
I'd really appreciate it if you can provide more details about the TLS/SSL issue. I'd gladly help to get it prioritized at the OpenJDK side.

cstamas
Tuesday at 8:45 PM
I cannot sadly, it just happens on CI, on GH action to be more precise, for example here https://github.com/apache/maven/actions/runs/17913416241/job/50929944116
8:46
but, due resolver codebase, you cannot see HttpClient in stack traces, only the message coming from it
8:46
Caused by: org.apache.maven.api.services.model.ModelResolverException: The following artifacts could not be resolved: eu.maveniverse.maven.mimir:mimir:pom:0.7.8 (absent): Could not transfer artifact eu.maveniverse.maven.mimir:mimir:pom:0.7.8 from/to central (https://repo.maven.apache.org/maven2): Remote host terminated the handshake (remote repositories: central (https://repo.maven.apache.org/maven2, default, releases))
at org.apache.maven.impl.resolver.DefaultModelResolver.doResolveModel(DefaultModelResolver.java:160)
at org.apache.maven.impl.resolver.DefaultModelResolver.lambda$resolveModel$0(DefaultModelResolver.java:113)
👀
1

Romain
Tuesday at 8:54 PM
do you set tls version as system prop? (
https.protocols
) (edited)

Konrad Windszus
Tuesday at 9:51 PM
I think for https://bugs.openjdk.org/browse/JDK-8283544 there was a workaround to use another constructor used in our code, right @cstamas?

cstamas
Tuesday at 10:03 PM
fac1569
10:03
but there was this as well https://github.com/apache/maven-resolver/pull/1509/files
10:03
and this https://github.com/apache/maven-resolver/blob/master/maven-resolver-transport-jdk-[…]/main/java/org/eclipse/aether/transport/jdk/JdkTransporter.java
10:05
@Romain no prop set, all is "vanilla" on GH CI like you have locally
10:06
the TLS issues are intermittent and that is why we never could reproduce them (or when happen, we have no any debug log, as that would clog logging)

Romain
Tuesday at 10:13 PM
can be worth trying, there are 4-5 props more or less official/hidden which help
10:13
sadly some must be set on the jvm otherwise it comes likely too late

Matthias Bünger
Wednesday at 2:27 PM
We have the same thing at my work. We would like to use the JDK one, but that does not support custom verifier

Volkan Yazıcı
Wednesday at 2:29 PM
@Matthias Bünger Mind elaborating on what you mean with "[Java HttpClient] does not support custom verifier", please?

Matthias Bünger
Wednesday at 2:32 PM
We need to disable hostname-verifier on a client base (it‘s only possible globally via system property). That‘s why we also use the Apache one

Romain
Wednesday at 2:38 PM
We would like to use the JDK one, but that does not support custom verifier
I'm pretty sure this is not true but the API changed IIRC and is lower level (edited)

cstamas
Wednesday at 2:39 PM
is global only https://bugs.openjdk.org/browse/JDK-8213309

Romain
Wednesday at 2:43 PM
the system prop, not the impl
2:44
in your ssl context i think you can set sslParameters.setEndpointIdentificationAlgorithm to disable it
2:44
(makes almost 2 years I didn't do it again but there is something like that to do it)

Volkan Yazıcı
Wednesday at 3:03 PM
@Matthias Bünger @Romain @cstamas Thanks for the valuable feedback! I've communicated these to the net-dev crew. If you have any further remarks, pain (or nice?) points to share, I'd be more than happy to hear.
🍻
2

Christoph Läubrich
Friday at 12:43 PM
@Volkan Yazıcı Tycho suffers from the same problems (mainly HTTP GOAWAY) what is a real pita ...).
In general I think we also noticed missing NTLM support and proxy auth let me check...
12:43
Beside that I found this one quite blocking wen it comes to switching protocols: https://bugs.java.com/bugdatabase/view_bug?bug_id=JDK-8361305
bugs.java.com
Bug ID: JDK-8361305 WebSocket.Builder should support HTTP Upgrades to a Websocket Connection
Component: core-libs | Sub-Component: java.net
12:44
Found it:

• org.eclipse.ecf.provider.filetransfer.httpclientjava does not support proxy authentication eclipse-ecf/ecf#76
  12:46
  Theres also this one https://stackoverflow.com/questions/69320901/jdk-http-client-ntlm
  Stack OverflowStack Overflow
  jdk http client ntlm
  I am not by far an expert in different protocols over http, but I can do this with curl to one of the APIs at my workplace:
  curl --http1.1 --ntlm -u 'user:pass' 'MY_URL'
  This works just fine. But

cstamas
Friday at 12:47 PM
isn't NTLM being dropped by MS as well? Also apache client5 dropped support for it AFAIK (edited)

Christoph Läubrich
Friday at 12:49 PM
I jsut knwo some users (mostly bigger companiees) complain when NTLM/Kerberos whatever they use 😄
12:49
I personally have no use for it 😅
12:50
But ist often the legacy that hinders adoption of new technologies.
12:51
Also at some kind it seems HttpUrlConnection has support for it so I think its not smth completeley new to the JDK

cstamas
Friday at 12:52 PM
that class is java 1.0. no? 😄

Christoph Läubrich
Friday at 12:52 PM
:man-shrugging:
12:53
So as long as pache-http offers "more" support its hard to argue to use the builtin if it lacks features even if they are only used in a small cases

cstamas
Friday at 12:54 PM
JDK client is not by any means "apache client replacement" 😄 (IMHO at least) (edited)

Christoph Läubrich
Friday at 12:59 PM
Well question was how to make it the default, what makes it a replacement somehow 😉

Romain
Friday at 3:39 PM
well thing is both have drawbacks and will bother users, jre one is built in as only advantage but it is a big one so 49-51 in favor of jre one for me

Volkan Yazıcı
Friday at 3:49 PM
@Christoph Läubrich Thanks so much! I've shared all these remarks with the HttpClient maintainers. Much appreciated! 🙇