8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default #7688

Michael-Mc-Mahon · 2022-03-04T09:37:21Z

Hi,

Could I get the following change reviewed please, which is to disable the MD5 message digest algorithm by default in the HTTP Digest authentication mechanism? The algorithm can be opted into by setting a new system property "http.auth.digest.reEnabledAlgs" to include the value MD5. The change also updates the Digest authentication implementation to use some of the more secure features defined in RFC7616, such as username hashing and additional digest algorithms like SHA256 and SHA512-256.

Michael

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed
Change requires a CSR request to be approved

Issues

JDK-8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default
JDK-8282649: Disable http DIGEST mechanism with MD5 by default (CSR)

Reviewers

Weijun Wang (@wangweij - Reviewer) ⚠️ Review applies to c55fdd9
Daniel Fuchs (@dfuch - Reviewer) ⚠️ Review applies to b2095e0

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7688/head:pull/7688
$ git checkout pull/7688

Update a local copy of the PR:
$ git checkout pull/7688
$ git pull https://git.openjdk.java.net/jdk pull/7688/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7688

View PR using the GUI difftool:
$ git pr show -t 7688

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7688.diff

Michael-Mc-Mahon · 2022-03-04T09:37:43Z

/csr needed

bridgekeeper · 2022-03-04T09:38:51Z

👋 Welcome back michaelm! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2022-03-04T09:40:02Z

@Michael-Mc-Mahon has indicated that a compatibility and specification (CSR) request is needed for this pull request.
@Michael-Mc-Mahon this pull request must refer to an issue in JBS to be able to link it to a CSR request. To refer this pull request to an issue in JBS, please use the /issue command in a comment in this pull request.

openjdk · 2022-03-04T09:41:33Z

@Michael-Mc-Mahon The following label will be automatically applied to this pull request:

net

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

Michael-Mc-Mahon · 2022-03-04T10:30:43Z

I have still to add the doc update describing the system property, which will come shortly. I may suggest a change of name to the system property to better reflect its exact meaning/purpose

mlbridge · 2022-03-04T11:12:25Z

Webrevs

dfuch · 2022-03-04T11:25:38Z

Should we instead have a property to disable algorithms, whose default value would contain "MD5" by default?

Michael-Mc-Mahon · 2022-03-04T12:03:44Z

Should we instead have a property to disable algorithms, whose default value would contain "MD5" by default?

I considered that and implemented it that way at the start, but what you would end up with then is users running their code with something like: -DdisabledAlgNames=""

I find that style leads to a much less explicit "opting in" than by making the user explicitly identify the deprecated algorithm by name.

dfuch · 2022-03-04T12:12:25Z

I considered that and implemented it that way at the start, but what you would end up with then is users running their code with something like: -DdisabledAlgNames=""

I find that style leads to a much less explicit "opting in" than by making the user explicitly identify the deprecated algorithm by name.

Right - but it would also allow users to opt-in to disable more algorithms by listing them in the property

Michael-Mc-Mahon · 2022-03-04T12:19:07Z

I considered that and implemented it that way at the start, but what you would end up with then is users running their code with something like: -DdisabledAlgNames=""
I find that style leads to a much less explicit "opting in" than by making the user explicitly identify the deprecated algorithm by name.

Right - but it would also allow users to opt-in to disable more algorithms by listing them in the property

In practical terms, the only other likely candidate there is SHA-1. If that weren't the case, I'd disagree with your point.

So, maybe, we could have a 2nd net property with the default disabled algorithms and in net.properties we identify MD5 only for now. Users could add to that list if they want or even specify it on the command line. I think it's potentially confusing, but maybe there is a case for adding to the disabled list. I need to think about a way to do this without subvertng the point about making the user explicitly opt in.

Michael-Mc-Mahon · 2022-03-04T12:29:28Z

I considered that and implemented it that way at the start, but what you would end up with then is users running their code with something like: -DdisabledAlgNames=""
I find that style leads to a much less explicit "opting in" than by making the user explicitly identify the deprecated algorithm by name.

Right - but it would also allow users to opt-in to disable more algorithms by listing them in the property

In practical terms, the only other likely candidate there is SHA-1. If that weren't the case, I'd disagree with your point.

So, maybe, we could have a 2nd net property with the default disabled algorithms and in net.properties we identify MD5 only for now. Users could add to that list if they want or even specify it on the command line. I think it's potentially confusing, but maybe there is a case for adding to the disabled list. I need to think about a way to do this without subvertng the point about making the user explicitly opt in.

Thinking about it again, I wonder if we should just deprecate SHA-1 at the same time. I think there will be less compatibility impact than with MD5, and it's basically broken as well. I don't see a reason to opt out of other algorithms at this time.

dfuch · 2022-03-04T13:06:13Z

So, maybe, we could have a 2nd net property with the default disabled algorithms and in net.properties we identify MD5 only for now. Users could add to that list if they want or even specify it on the command line. I think it's potentially confusing, but maybe there is a case for adding to the disabled list. I need to think about a way to do this without subvertng the point about making the user explicitly opt in.

Thinking about it again, I wonder if we should just deprecate SHA-1 at the same time. I think there will be less compatibility impact than with MD5, and it's basically broken as well. I don't see a reason to opt out of other algorithms at this time.

I see - maybe we should have a security property identifying the list of algorithm that are disabled, and then a system property to reenable them?

dfuch · 2022-03-04T13:13:47Z

src/java.base/share/classes/sun/net/www/protocol/http/DigestAuthentication.java

+    private static final Set<String> defDisabledAlgs =
+        Set.of("MD5");


What I'm suggesting is that the content of this set could be seeded with the value of a Security Property, defined in java.security - rather than have the default value hardcoded here.

Yes, I think that should work

jaikiran · 2022-03-04T13:50:37Z

src/java.base/share/classes/java/net/doc-files/net-properties.html

@@ -224,6 +224,12 @@ <H2>Misc HTTP URL stream protocol handler properties</H2>
 	      property is defined, then its value will be used as the domain
 	      name.</P>
 	</OL>
+	<LI><P><B>{@systemProperty http.auth.digest.reEnabledAlgs}</B> (default: &lt;none&gt;)<BR>


Hello Michael, from what I understand of doc-files directory (which is where this html file resides) the javadoc tool considers it an unprocessed files location[1]. So would adding the systemProperty javadoc taglet here be necessary? I think this won't end up being listed in the system properties index generated by the javadoc tool and I think this line here will just get rendered literally.

[1] https://docs.oracle.com/javase/8/docs/technotes/tools/windows/javadoc.html

This actually seems to work. If you build the docs for JDK mainline, you can search for instance, for http.keepAlive.time.proxy in the javadoc search box and it will lead you net-properties.html.

You are right indeed. I just checked the generated system properties index and it does list these properties and even links back to this document. In fact, I even found a mail which clearly states that this processing of @systemProperties is supported for doc-files https://mail.openjdk.java.net/pipermail/core-libs-dev/2018-November/056653.html

jaikiran · 2022-03-04T14:02:42Z

src/java.base/share/classes/sun/net/www/protocol/http/DigestAuthentication.java

+            if (logger.isLoggable(PlatformLogger.Level.INFO)) {
+                logger.info(msg + " This constraint may be relaxed by setting " +
+                     "the \"http.auth.digest.enabledDigestAlgs\" system property.");
+            }


I suspect this is a typo and perhaps should have been http.auth.digest.reEnabledAlgs?

Thanks. I'm going to change the name once more and will update it then.

mlbridge · 2022-03-04T14:06:22Z

Mailing list message from Bernd Eckenfels on net-dev:

Hello,

While I like the idea of the user having to explicitely specify the rexenabled legacy algorithms (as opposed to removing the defaultsdisabled) it is not the style the other algorithm policies in JCE work - so it might be confusing.

But, more critically I would separate the enabling/implementing of new algorithms from disabling old ones. Especially since there needs to be changes on the server side first. (And I wonder if this can be negotiated anyway?).

So why not start with a ?provide new DIGEST Mechanisms? change? Having said that, would it need to start out with disabled new mechanisms so the update won?t change the behavior? (If there is no negotiation?)

Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
Von: net-dev <net-dev-retn at openjdk.java.net> im Auftrag von Michael McMahon <michaelm at openjdk.java.net>
Gesendet: Friday, March 4, 2022 1:33:06 PM
An: net-dev at openjdk.java.net <net-dev at openjdk.java.net>
Betreff: Re: RFR: 8281561: Disable http DIGEST mechanism with MD5 by default

On Fri, 4 Mar 2022 09:37:21 GMT, Michael McMahon <michaelm at openjdk.org> wrote:

Hi,

Could I get the following change reviewed please, which is to disable the MD5 message digest algorithm by default in the HTTP Digest authentication mechanism? The algorithm can be opted into by setting a new system property "http.auth.digest.reEnabledAlgs" to include the value MD5. The change also updates the Digest authentication implementation to use some of the more secure features defined in RFC7616, such as username hashing and additional digest algorithms like SHA256 and SHA512-256.

- Michael

I considered that and implemented it that way at the start, but what you would end up with then is users running their code with something like: -DdisabledAlgNames=""
I find that style leads to a much less explicit "opting in" than by making the user explicitly identify the deprecated algorithm by name.

Right - but it would also allow users to opt-in to disable more algorithms by listing them in the property

In practical terms, the only other likely candidate there is SHA-1. If that weren't the case, I'd disagree with your point.

So, maybe, we could have a 2nd net property with the default disabled algorithms and in net.properties we identify MD5 only for now. Users could add to that list if they want or even specify it on the command line. I think it's potentially confusing, but maybe there is a case for adding to the disabled list. I need to think about a way to do this without subvertng the point about making the user explicitly opt in.

Thinking about it again, I wonder if we should just deprecate SHA-1 at the same time. I think there will be less compatibility impact than with MD5, and it's basically broken as well. I don't see a reason to opt out of other algorithms at this time.

-------------

PR: https://git.openjdk.java.net/jdk/pull/7688
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/net-dev/attachments/20220304/0dd0e778/attachment.htm>

src/java.base/share/classes/sun/net/www/protocol/http/DigestAuthentication.java

wangweij

LGTM now. It will be even nicer if the known answer tests in RFC 7616 can be included.

seanjmullan · 2022-03-15T20:43:30Z

Suggest changing the bug title to reflect that SHA-1 is now also disabled by default, i.e. "Disable http DIGEST mechanisms with MD5 or SHA-1 by default".

openjdk · 2022-03-23T04:56:18Z

@Michael-Mc-Mahon This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default

Reviewed-by: weijun, dfuchs

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 5 new commits pushed to the master branch:

0c472c8: 8283756: (zipfs) ZipFSOutputStreamTest.testOutputStream should only check inflated bytes
d6fa8b0: 8283469: Don't use memset to initialize members in FileMapInfo and fix memory leak
8567266: 8283683: Make ThreadLocalRandom a final class
f4eaa16: 8283728: jdk.hotspot.agent: Wrong location for RISCV64ThreadContext.java
cdef087: 8283727: P11KeyGenerator has import statement with two semicolons after JDK-8267319

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

Michael-Mc-Mahon · 2022-03-25T15:10:51Z

webrevs 06 & 07 have the latest changes, if anyone is just looking at the incremental changes.

The main changes are new tests including the hardcoded test from RFC7616 (which has to be run using a modified JDK).
Also, some small reorganisation of the DigestAuthentication impl code.

dfuch · 2022-03-25T17:21:11Z

src/java.base/share/classes/sun/net/www/protocol/http/DigestAuthentication.java

-        if (session) {
-            algorithm = algorithm.substring(0, algorithm.length() - 5);
-        }
+        boolean session = algorithm.endsWith ("-sess");


should that be digest.endsWith("-sess"); ?

No, the digest field refers to the actual message digest algorithm (as known to the security libraries). The algorithm field holds the algorithm name as it is defined in RFC7616.

I am confused here - because you converted algorithm to upper case, so it should never end with -sess?

Look at line 478: The algorithm field is reset here to be the upper case of the digest name plus the -sess suffix in lower case.

Doh! Somehow I'd read that as digestName = digestName + "-sess".
All clear now.

dfuch · 2022-03-28T11:25:02Z

src/java.base/share/classes/sun/net/www/protocol/http/DigestAuthentication.java

-        if (session) {
-            algorithm = algorithm.substring(0, algorithm.length() - 5);
-        }
+        boolean session = algorithm.endsWith ("-sess");


Doh! Somehow I'd read that as digestName = digestName + "-sess".
All clear now.

Michael-Mc-Mahon · 2022-03-28T13:50:48Z

/integrate

openjdk · 2022-03-28T13:52:14Z

Going to push as commit 7f2a3ca.
Since your change was applied there have been 5 commits pushed to the master branch:

0c472c8: 8283756: (zipfs) ZipFSOutputStreamTest.testOutputStream should only check inflated bytes
d6fa8b0: 8283469: Don't use memset to initialize members in FileMapInfo and fix memory leak
8567266: 8283683: Make ThreadLocalRandom a final class
f4eaa16: 8283728: jdk.hotspot.agent: Wrong location for RISCV64ThreadContext.java
cdef087: 8283727: P11KeyGenerator has import statement with two semicolons after JDK-8267319

Your commit was automatically rebased without conflicts.

openjdk · 2022-03-28T13:52:32Z

@Michael-Mc-Mahon Pushed as commit 7f2a3ca.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Michael-Mc-Mahon added 9 commits February 23, 2022 11:59

furst part of impl

f7549d3

update

d4f0c10

Merge branch 'master' into md5

b602abf

added test

2564ea2

fixed problem in copyright header

683cec4

added userhash support plus test

8d2f335

Merge branch 'master' into md5

960d744

fixed up existing tests using digest auth

2d62ff1

fixed one more test

0ab8447

openjdk bot added the csr Pull request needs approved CSR before integration label Mar 4, 2022

openjdk bot added the net net-dev@openjdk.org label Mar 4, 2022

Michael-Mc-Mahon added 2 commits March 4, 2022 10:37

update property name. add documentation

77b2564

fix whitespace

f0fb72d

openjdk bot added the rfr Pull request is ready for review label Mar 4, 2022

dfuch reviewed Mar 4, 2022

View reviewed changes

jaikiran reviewed Mar 4, 2022

View reviewed changes

Michael-Mc-Mahon added 3 commits March 14, 2022 10:14

removed swp file

d59aa92

update after third review round

65d549f

Merge branch 'master' into md5

c55fdd9

dfuch reviewed Mar 14, 2022

View reviewed changes

src/java.base/share/classes/sun/net/www/protocol/http/DigestAuthentication.java Outdated Show resolved Hide resolved

wangweij approved these changes Mar 14, 2022

View reviewed changes

made disabledAlgorithms immutable

66c320c

dfuch approved these changes Mar 15, 2022

View reviewed changes

Michael-Mc-Mahon added 3 commits March 21, 2022 10:41

incomplete test update

0ed4486

delete .swp file

62aeeb0

Merge branch 'master' into md5

55bcd0d

openjdk bot removed the csr Pull request needs approved CSR before integration label Mar 23, 2022

openjdk bot added the ready Pull request is ready to be integrated label Mar 23, 2022

latest update

2ef5636

Michael-Mc-Mahon changed the title ~~8281561: Disable http DIGEST mechanism with MD5 by default~~ 8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default Mar 25, 2022

forgot update to DigestAuth test

b2095e0

Merge branch 'master' into md5

255055d

dfuch reviewed Mar 25, 2022

View reviewed changes

merge branch 'master' into md5

946142f

dfuch approved these changes Mar 28, 2022

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Mar 28, 2022

openjdk bot closed this Mar 28, 2022

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Mar 28, 2022

Michael-Mc-Mahon deleted the md5 branch March 28, 2022 14:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default #7688

8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default #7688

Michael-Mc-Mahon commented Mar 4, 2022 •

edited by openjdk bot

Loading

Michael-Mc-Mahon commented Mar 4, 2022

bridgekeeper bot commented Mar 4, 2022

openjdk bot commented Mar 4, 2022

openjdk bot commented Mar 4, 2022

Michael-Mc-Mahon commented Mar 4, 2022

mlbridge bot commented Mar 4, 2022 •

edited

Loading

dfuch commented Mar 4, 2022

Michael-Mc-Mahon commented Mar 4, 2022

dfuch commented Mar 4, 2022 •

edited

Loading

Michael-Mc-Mahon commented Mar 4, 2022 •

edited

Loading

Michael-Mc-Mahon commented Mar 4, 2022

dfuch commented Mar 4, 2022

dfuch Mar 4, 2022

Michael-Mc-Mahon Mar 4, 2022

jaikiran Mar 4, 2022 •

edited

Loading

dfuch Mar 4, 2022

jaikiran Mar 4, 2022

jaikiran Mar 4, 2022

Michael-Mc-Mahon Mar 4, 2022

mlbridge bot commented Mar 4, 2022

wangweij left a comment

seanjmullan commented Mar 15, 2022

openjdk bot commented Mar 23, 2022 •

edited

Loading

Michael-Mc-Mahon commented Mar 25, 2022 •

edited

Loading

dfuch Mar 25, 2022

Michael-Mc-Mahon Mar 28, 2022 •

edited

Loading

dfuch Mar 28, 2022

Michael-Mc-Mahon Mar 28, 2022

dfuch Mar 28, 2022

dfuch Mar 28, 2022

Michael-Mc-Mahon commented Mar 28, 2022

openjdk bot commented Mar 28, 2022

openjdk bot commented Mar 28, 2022

		private static final Set<String> defDisabledAlgs =
		Set.of("MD5");

8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default #7688

8281561: Disable http DIGEST mechanism with MD5 and SHA-1 by default #7688

Conversation

Michael-Mc-Mahon commented Mar 4, 2022 • edited by openjdk bot Loading

Progress

Issues

Reviewers

Reviewing

Michael-Mc-Mahon commented Mar 4, 2022

bridgekeeper bot commented Mar 4, 2022

openjdk bot commented Mar 4, 2022

openjdk bot commented Mar 4, 2022

Michael-Mc-Mahon commented Mar 4, 2022

mlbridge bot commented Mar 4, 2022 • edited Loading

Webrevs

dfuch commented Mar 4, 2022

Michael-Mc-Mahon commented Mar 4, 2022

dfuch commented Mar 4, 2022 • edited Loading

Michael-Mc-Mahon commented Mar 4, 2022 • edited Loading

Michael-Mc-Mahon commented Mar 4, 2022

dfuch commented Mar 4, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jaikiran Mar 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mlbridge bot commented Mar 4, 2022

wangweij left a comment

Choose a reason for hiding this comment

seanjmullan commented Mar 15, 2022

openjdk bot commented Mar 23, 2022 • edited Loading

Michael-Mc-Mahon commented Mar 25, 2022 • edited Loading

Choose a reason for hiding this comment

Michael-Mc-Mahon Mar 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Michael-Mc-Mahon commented Mar 28, 2022

openjdk bot commented Mar 28, 2022

openjdk bot commented Mar 28, 2022

Michael-Mc-Mahon commented Mar 4, 2022 •

edited by openjdk bot

Loading

mlbridge bot commented Mar 4, 2022 •

edited

Loading

dfuch commented Mar 4, 2022 •

edited

Loading

Michael-Mc-Mahon commented Mar 4, 2022 •

edited

Loading

jaikiran Mar 4, 2022 •

edited

Loading

openjdk bot commented Mar 23, 2022 •

edited

Loading

Michael-Mc-Mahon commented Mar 25, 2022 •

edited

Loading

Michael-Mc-Mahon Mar 28, 2022 •

edited

Loading