SOLR-15843 Update Log4J to 2.15

[madrob]

https://issues.apache.org/jira/browse/SOLR-15843

[ramenjosh]

Hey team, can this also be backported to 7.7 as a new 7.7.4 release? Many people still running Solr 7 will also require this fix.

[janhoy]

Looks good, you can consider moving the changes entry to bugfix as our reason to upgrade is not to use new features but to get bugfixes...

[uschindler]

Hey team, can this also be backported to 7.7 as a new 7.7.4 release? Many people still running Solr 7 will also require this fix.

This won't happen anymore as 7.x is out of maintenance. To fix it, just modify your local solr.in.sh and pass the extra command line parameter: -Dlog4j2.formatMsgNoLookups=true

We will add this to the release notes.

[uschindler]

Please add the above instructions to release notes, so we inform users how to fix older versions (like 7.x, 6.x,...).

[madrob]

Release notes or the security page on the site (or both?)

Solr 7 is affected but AFAICT Solr 5 and 6 are not because they use log4j 1.2.17

[uschindler]

Release notes or the security page on the site (or both?)

Solr 7 is affected but AFAICT Solr 5 and 6 are not because they use log4j 1.2.17

I would maybe do both. Let's add a notice with the fix to our news list (under security) and add the release note later when we publish a release.

In addition, depending on your configuration (not the default), log4j v1 is also affected. Theres a special appender doing the same style of JNDI stuff: apache/logging-log4j2#608 (comment)

[plumdog]

Should we expect patched releases for older 8.x versions? In my case, 8.9? Or will I have to take 8.11?

[plumdog]

Also, regarding Solr 5 and 6 and log4j: apache/logging-log4j2#608 (comment)

Based on that, I think they would be affected.

[madrob]

Should we expect patched releases for older 8.x versions? In my case, 8.9? Or will I have to take 8.11?

You should not expect additional 8.9.z releases

[uschindler]

Also, regarding Solr 5 and 6 and log4j: apache/logging-log4j2#608 (comment)

This is only a problem if you use special appenders, which solr does not do.

[acdha]

I was wondering whether someone could confirm that the top-level version lock will also affect the contrib prometheus-exporter. My scan of our container images turned up a second copy of log4j 2.14.1 in /opt/solr-8.11.0/contrib/prometheus-exporter/lib/.

[plumdog]

You should not expect additional 8.9.z releases

@madrob So if I want something both stable and patched, I'll need 8.11.next?

[uschindler]

You should not expect additional 8.9.z releases

@madrob So if I want something both stable and patched, I'll need 8.11.next?

I would NOT wait for a release or asking the same question like this one all the time (ask them one time on mailing list please) and instead act NOW:

Just open your solr.in.sh in your favorite editor and add SOLR_OPTS="$SOLR_OPTS -Dlog4j2.formatMsgNoLookups=true". Restart Solr. Done, all safe.

[uschindler]

You should not expect additional 8.9.z releases

@madrob So if I want something both stable and patched, I'll need 8.11.next?

I would NOT wait for a release or asking the same question like this one all the time (ask them one time on mailing list please) and instead act NOW:

Just open your solr.in.sh in your favorite editor and add SOLR_OPTS="$SOLR_OPTS -Dlog4j2.formatMsgNoLookups=true". Restart Solr. Done, all safe.

Here is the security advisory, it shows you how to harden your solr server: https://solr.apache.org/security.html#apache-solr-affected-by-apache-log4j-cve-2021-44228

[mario-canva]

The Apache log4j security advisory was updated recently stating the flag -Dlog4j2.formatMsgNoLookups=true is not a sufficient mitigation for log4j versions below 2.15.0. Which is the case for Solr 7.7.3 and below, so the Solr advisory also needs to be updated.

What other possible mitigations we can use for Solr 7.7.3? Any chances to get a patch for this version as well? I know it is stated out of maintenance but a patch it would really go a long way in helping people to mitigate this vulnerability. Let me know if create a PR helps in getting a patch moving and I will raise one.

[uschindler]

Hi,

Solr does use MDC (the %X pattern), but the values are not user generated and all come from config files and are enforced to comply to certain formats (e.g., no $ possible). Shard, replica, collection names are sanitized.

In short all fine, no need to change the mitigation instructions. There is also no need to update log4j in older versions of Solr.

Uwe

[mario-canva]

Thanks @uschindler appreciate the quick response! However, their advisory also states other attack vectors may be possible:

The reason these measures are insufficient is that, in addition to the Thread Context attack vector mentioned above, there are still code paths in Log4j where message lookups could occur: known examples are applications that use Logger.printf("%s", userInput), or applications that use a custom message factory, where the resulting messages do not implement StringBuilderFormattable. There may be other attack vectors.

At the moment we are going with the mitigation they suggested here:

remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

Although, if we could have a patch for 7.7.3 to upgrade the log4j version for 2.16.0 that would be best.

[uschindler]

The other attack vectors are also not possible with Solr:

• Logger.printf("%s", userInput) is not used
• custom message factory is not used

Uwe

[mario-canva]

Understood. I still think pays off being a bit conservative here, it is a critical CVE after all and the log4j team stated other attack vectors may be possible (beyond the ones they have found already).

Internally we will follow Log4j advice to not use that flag as mitigation and remove the JndiLookup class from the jar.

Thanks!