Migrate Ebean to use of JDK System.Logger (make SLF4J-API an optional dependency)

[rbygrave]

In case people are unaware Java 9 introduced System.Logger.

Ebean and all its dependencies (ebean-datasource, avaje-config, avaje classpath-scanner etc) have been using slf4j-api and that has been the good and sensible thing to do.

However, in a world of both classpath and module-path with some people using module-path (with module-info etc) sticking to slf4j-api might not be the best long term path. More specifically the module-path compatible versions of slf4j and logback etc have been in alpha for many many years and it seems problematic for people who want to be using module-path and don't want to depend on alpha level dependencies.

As a library, Ebean and these avaje libraries should ideally support projects wanting to be in either module-path world or classpath world. Having had various discussion on this in other places I have started to move the avaje libraries from using slf4j-api to use System.Logger instead. In addition we provide a bridge https://github.com/avaje/slf4j-jdk-platform-logging.

• If you are not familiar with System.Logger please go have a look
• If people are unhappy about the use/adoption of System.Logger please provide feedback.
• If you want System.Logger to forward to slf4j-api 1.7.x look to use https://github.com/avaje/slf4j-jdk-platform-logging

The intention is that adoption of System.Logger will start with avaje-config and avaje classpath-scanner, then ebean-datasource ... and then we will see where we are at. These libraries don't do much in terms of logging.

My gut feeling is that Ebean is unfortunately a relatively early mover in terms of System.Logger adoption and I feel that people are relatively unaware of System.Logger so I suspect this might lead to some confusion. Hopefully that is short lived and people generally agree that libraries should be as agnostic as they can be wrt logging and hence libraries are somewhat expected to gradually move to System.Logger.

For Ebean itself to become agnostic of slf4j-api we would have some work and investigation to do (background execution is MDC aware etc) and I see that as some future followup step that we could discuss in a month or so once we have seen how this first part goes.

[rbygrave]

For avaje classpath-scanner, this PR #2648 ... bumps to a version that uses System.Logger

[rbygrave]

To probably state the obvious, it's only because Ebean is now at Java 11 that we can do such a migration towards System.Logger. I expect as more libraries move to Java 9+ similar migrations to System.Logger will occur in the java ecosystem.

Link to similar issues/discussions:

• micronaut - Move from SLF4j to Java 9+ System.Logger and provide binding for logback out of the box micronaut-projects/micronaut-core#7043
• jooq - Replace slf4j by System.Logger jOOQ/jOOQ#12764
• spring - Investigate possible use of Java System Logger over Commons Logging API spring-projects/spring-framework#27810
• avaje-inject - Dependency on SLF4J - Migrate to System.Logger avaje/avaje-inject#156

[rbygrave]

More Background

comments about System.Logger by Stuart Marks on Twitter: https://mobile.twitter.com/stuartmarks/status/1254819581585047552

It's mainly for the JDK itself, but it can also be used by anything that doesn't want to add a dependency for a logging facade.

The primary consumer is the JDK, but it’s public so anyone can use it. It seems useful for libraries that want to minimize dependencies.

That said, if one can live within the limitations of System.Logger, it's public, so it's fine for anyone to use.

... and discussions on reddit at:
https://www.reddit.com/r/java/comments/rdv98z/have_you_ever_wondered_how_javas_logging/

... and avaje-inject issue:
avaje/avaje-inject#156 - Dependency on SLF4J - Migrate to System.Logge

[rbygrave]

Ebean 13.6.0 has the underlying libraries bumped to use java platform logger - System.Logger.

I'll change this ticket now to that of changing Ebean itself to use System.Logger or more accurately make slf4j-api an optional dependency (with the MDC use for Background execution).

[federicorispo]

I'm interested on this migration because if a library can be agnostic to what logger is used then it will be a win win situation.
However I'm concerned about this quote:

That said, if one can live within the limitations of System.Logger, ...

What are these limitations?

[rbygrave]

limitations of System.Logger

As I see it the limitation relative to slf4j-api is that System.Logger does not provide MDC (Mapped diagnostic context - https://logback.qos.ch/manual/mdc.html). I'd argue that libraries themselves don't need MDC.

Note that the application can still choose to use MDC, it's just that the library can't use MDC.

[rbygrave]

Currently AWS Lambda runtime is NOT service loading a System.LoggerFinder.

That is, without a System.LoggerFinder the JDK will default the System.Logger implementations to use JUL - Java Util Logging. If we don't want that, the current practice is to add io.avaje:avaje-slf4j-jpl as a dependency - which provides a System.LoggerFinder implementation that adapts to SLF4J-API. That all works great except ... when running in AWS Lambda Java runtime (and at this stage I don't know why it does not work in lambda). The other places we might not be able to control the System.Logger implementation is in JEE containers and Servlet containers (war/ear deployments).

Now, if we have logging of System.Logger going to JUL then we can either use the jul-to-slf4j bridge ... or use JUL ... or modify the plan to introduce one extra layer of indirection that will work when service loading a System.LoggerFinder does not work (like AWS Lambda).

I'm pretty sure I'm going to introduce one extra layer of indirection. This means that we are still using System.Logger but we have 2 options for controlling the implementation being used. Option 1. Use the standard System.LoggerFinder (doesn't work in lambda, probably doesn't work in JEE or Servlet containers) 2. Use a new AppLog.Provider (our extra layer of indirection, and defaults to just using System.getLogger() but gives apps the ability to control the implementation in these environments where jvm startup isn't under application control.

The plan:

• Create AppLog (a library with 2 classes in it, AppLog and AppLog.Provider)
• Use System.Logger logger = AppLog.getLogger("foo"); instead of System.Logger logger = System.getLogger("foo");
• AppLog defaults to just calling System.getLogger(...)
• Provide a AppLog.Provider implementation that adapts to slf4j-api 1.7.x
• Modify the avaje and ebean libraries to use AppLog.getLogger(...);

Not ideal per say but we do this because dealing with JUL can be a PITA ...

[jnehlmeier]

Given that you encounter some troubles and trying to workaround them (which makes things less clean):

Are you aware that SLF4J 2.0 is a drop-in replacement which is fully backwards compatible. The only thing you need to be aware of is that if you use SLF4J 2.0 you also need to update the logging implementation because 2.0 uses ServiceLoader which 1.7.x does not.

So from ebean point of view it would be totally fine to still depend on SLF4J and users of SLF4J can switch to 2.0 without any issue if they want to use module-path. So to be super correct you could define a minimal version of SLF4J that ebean requires, which would be 1.7.x.

https://www.slf4j.org/faq.html#changesInVersion200

[rbygrave]

Slf4j 2.x has been alpha for more than 3 years. The only project I have seen depend on it is jetty.

Are you using 2.x?

How do you feel about the alpha status of 2.x?

The concern is the shear length of time in alpha and it has significantly reduced my confidence that slf4j-api 2.x is actually going to be the successor to slf4j-api 1.x. Heck, I talked to Ceki about a platform adapter for 1.7.x and he wasn't interested so I took that on as avaje slf4j-jdk-platform-logging ... The question of when it will leave alpha status hasn't been answered for some time now (years).

In terms of System.Logger, I actually like it's simplicity as an API. I'm pretty comfortable that there is growing interest in it. No one has yet complained or commented about its use so far (which is interesting because I felt the default to JUL could have prompted some feedback).

Hmmm.

[jnehlmeier]

Slf4j 2.x has been alpha for more than 3 years. The only project I have seen depend on it is jetty.

Are you using 2.x?

How do you feel about the alpha status of 2.x?

Not yet using it because I haven't upgraded Jetty yet. But I will use it because I am pretty sure the alpha status is for the newly added fluent logging API. As stated the current 1.7.x API style will continue to work in 2.0 as it is a drop-in replacement.

And I am pretty sure jetty uses the same thinking and that is why they are fine using it to support module path in their newer jetty.

So an app can freely choose to upgrade to 2.0 even if libraries want 1.7.x.

The concern is the shear length of time in alpha and it has significantly reduced my confidence that slf4j-api 2.x is actually going to be the successor to slf4j-api 1.x.

See above. I am pretty sure it is because of the newly added fluent API.

In terms of System.Logger, I actually like it's simplicity as an API. I'm pretty comfortable that there is growing interest in it. No one has yet complained or commented about its use so far (which is interesting because I felt the default to JUL could have prompted some feedback).

I assume that apps will likely not use System.Logger because it is so limiting. So they will use some of the de-facto standards like SLF4J. Now if libraries start to move to System.Logger there is literally no difference for the app, except added complexity to configure System.Logger as well (next to jul, log4j, ...) to be redirected to SLF4J.

[jnehlmeier]

Just found qos-ch/slf4j#288 which also says that alpha has only been used because of the new, fluent API.

[rbygrave]

alpha has only been used because of the new, fluent API.

Nice.

alpha designation

You make great points. To a large extent the easy path for ebean is to stick with slf4j-api.

If ebean adopts System.Logger, the very next day Ceki will probably remove that alpha designation so it might be worth doing it for that reason :)

Some more factors:

• System.Logger might be limited but it's all a library needs (no need for MDC, Marker etc)
• logging for small libraries - avaje-config, avaje-classpath-scanner (+ avaje-inject, avaje-jex)
• have we over engineered logging? / lets make logging simpler? (tinylog, logging to sysout, logback is 800kb etc)
• does native-image change our view on logging? logging needs of cli apps, native-image etc
• have we fallen into a world of using logging for things that we shouldn't be using logging for?
• majority case of "dedicated jvm / embedded server", no longer using JEE/Servlet container with shared jvm

There is also the "crazy Rob" factor. I see logback totaling around 800kb and slf4j-api 2.x around 60kb and the majority of folks do not blink at those numbers ... where as I see "death by a thousand cuts" and I feel rightly or wrongly that we as jvm developers need to enable ourselves for the fight with golang. Along the lines of, we can't fight golang with "50MB of dependencies" - all our dependencies matter and all our dependencies need to fully justify themselves.

"Crazy Rob" sees hibernate-core-5.4.21.Final.jar is 7.3MB alone without including it's dependencies. "Crazy Rob" says ebean's future fight for the next 16 years isn't going to be with hibernate purely based on hibernates shear size. Ebean's major competitors going forward over the next 16 years are things that are much much lighter (row mappers basically). Rightly or wrongly that makes me a lot more sensitive to dependencies (I've been eyeing up our antlr dependency for a few years now).

---

When I put my avaje-config, avaje-inject hat on (not the ebean hat) I'm well aware that slf4j-api 2.x is around 60kb and avaje-config is 47kb (the logging api dependency is larger that the library itself). avaje-inject is 67kb (the logging api dependency is only a little bit smaller than my dependency injection library).

Edit: I believe that library authors creating small libraries will be pretty motivated to adopt System.Logger. People are becoming more aware of their dependencies, size, transitive dependencies, security footprint.

---

Can I have my cake and eat it too? Can I enable people to use "super light logging" AND make it transparent for the current vast majority of slf4j-api folks? I believe I can do that via this extra avaje-applog dependency which is 2 classes / 3.2kb + avaje-applog-slf4j adapter which is 4 classes / 7kb.

Right now I believe that it is worth having that independence from slf4j-api because:

• We can swap out the avaje-applog-slf4j dependency and go "super light logging"
• We can include avaje-applog-slf4j ... and transparently support the status quo (slf4j-api)
• ... plus I've got jaded waiting for that alpha designation to be removed

... BUT if slf4j-api 2.x sticks and it's a dependency everyone has going forward in the future, then this work has been largely academic (only valuable to the very few that want "super light logging").

[rbygrave]

Note that spring, micronaut and jooq all have tickets for this question/issue. Looking at those, none of them have moved or concluded what they will do yet but they are still interesting to read. IF any of those move to System.Logger that would be a fairly big marker for the wider community.

With those projects having not moved or concluded a path forward yet, it does mean that avaje and ebean are somewhat early adopters on this path which makes it harder and maybe somewhat this is boiling down to more a gut instinct call. There is that chance that avaje and ebean take this approach and other libraries either take years to follow or do not follow at all. I also feel I have a far more extreme position when it comes to dependencies and getting lighter and I suspect a lot more people are far more conservative - heck, I'm the guy that is using avaje-inject to remove spring completely so I'm clearly not mainstream.

Note that I do hear other people pushing System.Logger adoption on reddit and twitter etc so we are not completely out there on this. In theory I'm not completely crazy and instead just somewhat crazy.

[jnehlmeier]

Don't get me wrong: It is fine if a library uses System.Logger, however I just wanted to point out that module-path and alpha designation isn't a real problem as it seemed like that was the major reason to do so. So maybe it is worth spending time on other issues for the time being, if changing the logging within all your libraries is a significant pile of work.

Along the lines of, we can't fight golang with "50MB of dependencies"

Well well, even if you try to, Java will never win against golang in terms of memory consumption, just because of the JVM. Startup times might be fine using GraalVM, but that's about it.

[rbygrave]

Java will never win against golang in terms of memory consumption

We don't have to outright win, just need to be in the ballpark. A lot of memory consumption can be put down to library choice rather than the JVM itself. Generally speaking, that means large reflection based libs like Jackson need to be replaced typically with code generation (like what Kotlin is doing and what we get with avaje-jsonb).

Code generation via Java annotation processing is our build time half way house between "fully dynamic world" and "closed world native-image". I believe this is a big part of the change needed to get us into the ballpark (plus zero tolerance for dynamic proxies and zero tolerance for classpath scanning). Do these 3 things and we are basically in the ballpark, we also happen to be very well placed to go to native-image from there as we are no reflection and a lot less code to compile.

Startup times might be fine using GraalVM, but that's about it.

Startup times for servers are fine once we adopt compile time DI (avaje-inject, dagger2, manual wiring) because DI has such an impact on the CPU startup profile - that and dynamic proxies. Spring DI isn't competitive and in my view Micronaut DI also isn't competitive (without native-image) because unlike Dagger and avaje-inject it doesn't actually determine wiring order at build time hence it is also much slower and much bigger (anything that is bigger takes more jit time and more memory for compilation cache). I need to look at Arc (cdi-lite) again.

If a stack includes Spring DI, Jackson, Spring MVC, JAX-RS Jersey or RestEasy ... then that stack includes libs that can be replaced by build time code generation today. My expectation is set at around a 7MB stack in total size with pretty much zero reflection, zero dynamic proxies, zero classpath scanning.

I've been looking at this side of things for more than 4 years now and I've done a lot of experiments and work in this space. Although things like avaje-inject are much smaller than ebean size wise I don't see it being less important, in fact it could even be more important and impactful.

changing the logging within all your libraries is a significant pile of work.

Just to say I had already changed those libraries. avaje-inject was where this whole logging conversation and debate started and so they all changed first, ebean is the last the change.

[rbygrave]

Startup times

TLDR: 400 millicores is absolutely doable.

Also, just adding that the other day I had a discussion on reddit about running jvm apps in Kubernetes where someone said "stop this nonsense about running in 1000 millicores or less". Now I've had Java services running in production with 400 millicores absolutely fine and around 5 sec startup. To me, there are Java devs out there who really haven't looked at this problem in much detail and to some extent have given up competing in the low resource scenarios - I think these devs are just missing out and not fully understanding the options.