Library does not support Log4j 2

[hallihan]

Azure Databricks is moving to Log4j 2 as of version 11: https://docs.microsoft.com/en-us/azure/databricks/release-notes/runtime/11.0#log4j-is-upgraded-from-log4j-1-to-log4j-2

This will require major modifications to this library to maintain functionality

[swan001]

Any news regarding this? We cannot afford not to follow Databricks runtime upgrades and are not yet committed to Azure log analytics. Some clarification regarding the roadmap would allow us to move forward (with or without the library and / or log analytics).
Thanks a lot for the great contribution for the community.

[ViktorDronov]

I`m also interested in Dbr11 compatibility

[dbarrios83]

Following also this issue

[promisinganuj]

This a crucial library and can be a deciding factor for many customers. Any clarification regarding the roadmap and alternate options would help immensely.

[alessio-cuzzocrea]

Can't we rely on the bridge library (log4j-1.2-api.jar) that is still present on Databricks 11.3?

[aditya-anand0510]

When can we expect support for databricks runtime 11 and above please

[gfree76]

Piling on - we also need support for DBR 11. This is a major concern for us. Any suggestions on how to make this work would be much appreciated.

If anyone has expertise in this and can point me to which thread to pull, I can start looking into it.

[markshoesmith]

DBR 11+ support is required for us to consider this for any deployment now.
Given the Apache Log4J 2 sink for Azure Data Explorer recently released, is a update more feasible?

[fanxinms]

We also need support for DBR 11. We are concerning for this. Any workaround to solve this issue ?

[alessio-cuzzocrea]

I'm currently working on this, on my free time. I can't provide any ETA but if anyone is interested we can collaborate.

[alessio-cuzzocrea]

I've some news: I've been able to successfully run the package on spark3.3 and populate the SparkMetric_CL. I could house some help in understanding how to make it better and suitable for the community.

[nigelmeakins]

@hallihan
To echo @alessio-cuzzocrea comment above, the article below
https://logging.apache.org/log4j/2.x/manual/migration.html
mentions the use of the Log4j1 -> 2 bridge approach to easing migration. Can this please be investigated if not already done? This would keep this alive for future Databricks RTs, which is after all what we'd like to have when adopting this as a recommended pattern for monitoring.

[nigelmeakins]

I've some news: I've been able to successfully run the package on spark3.3 and populate the SparkMetric_CL. I could house some help in understanding how to make it better and suitable for the community.

@alessio-cuzzocrea are you able to share this anywhere please? I'd be interested in seeing the steps you undertook. Did you get this working with Dbr RT 11+ (sorry, you only mention Spark 3.3) ?