Java HttpUrlConnection blocking Sec-Fetch-* headers

[waiterlong]

I set headers like as followings:

dest.put("User-Agent", getWinCommonUserAgent());
dest.put("Accept", ACCEPT);
dest.put("Accept-Encoding", ACCEPT_ENCODING);
dest.put("Accept-Language", ACCEPT_LANGUAGE);
dest.put("Sec-Fetch-Site", SEC_FETCH_SITE); //cross-site
dest.put("Sec-Fetch-Dest", SEC_FETCH_DEST);
dest.put("Sec-Fetch-Mode", SEC_FETCH_MODE); // no-cors
dest.put("Sec-Fetch-User", SEC_FETCH_USER);
dest.put("Upgrade-Insecure-Requests", "1");
dest.put("Cache-Control", "max-age=0");
return dest;

Connection connect = Jsoup.connect(url);
connect.headers(dest);
connect.referrer("https://www.google.com/").followRedirects(true).ignoreContentType(true);
doc = connect.execute().parse();

But in the server, I print the request headers, there are no headers Sec-Fetch-* printed

accept:   text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding:   gzip, deflate
accept-language:   zh-CN,zh;q=0.8,en-US;q=0.6,en;q=0.5;q=0.4
upgrade-insecure-requests:   1
cache-control:   max-age=0
referer:   https://www.google.com/
host:   127.0.0.1:8040
connection:   keep-alive

Anybody can help me with this question?

[jhy]

I was surprised by this - turns out that Java is silently dropping these headers in HttpURLConnection.

(Decompiled code:)

    private boolean isRestrictedHeader(String var1, String var2) {
        if (allowRestrictedHeaders) {
            return false;
        } else {
            var1 = var1.toLowerCase();
            if (restrictedHeaderSet.contains(var1)) {
                return !var1.equals("connection") || !var2.equalsIgnoreCase("close");
            } else {
                return var1.startsWith("sec-");
            }
        }
    }

Need to figure out how to flip enable this:

allowRestrictedHeaders = (Boolean)AccessController.doPrivileged(new GetBooleanAction("sun.net.http.allowRestrictedHeaders"));

[jhy]

Oh it's just a system property - will set in a static method of Request. Library users need to be set these headers.

[jhy]

Thanks for pointing this out! Fixed, will be in 1.14.1.