Could the project switch to using log4j for logs?

[Asimov4]

I see a lot of logs printed to System.out or System.err.
Would it be possible to use a library like log4j http://logging.apache.org/log4j/2.x/ and use log.error, log.warning, log.info, log.debug instead?
That would make it easier for users of the StanfordCoreNLP to manage which logs should be printed by choosing the log level of the project.

[AngledLuffa]

My initial reaction is that it would be quite a lot of work for very little
early return for us... we certainly could discuss it again in the future,
of course.

Why specifically log4j?

John
On Sep 12, 2013 7:18 PM, "Asimov4" notifications@github.com wrote:

I see a lot of logs printed to System.out or System.err.
Would it be possible to use a library like log4j
http://logging.apache.org/log4j/2.x/ and use log.error, log.warning,
log.info, log.debug instead?
That would make it easier for users of the StanfordCoreNLP to manage which
logs should be printed by choosing the log level of the project.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1
.

[Asimov4]

I don't have any strong opinion about log4j. It is pretty standard and does the job well.
I'd be willing to contribute if that is something that you would agree to.

[reckart]

IMHO log4j would be a pretty bad choice, since it is a logging framework, not a logging API. Apache Commons Logging for example is split into an API and an implementation part. The default implementation can be replaced with e.g. a SLF4J bridge which can then log using any logging backend, including log4j. There is also a logging API built directly into Java: Java Utility Logging - it requires no extra dependencies and can likewise (although with a bit of extra effort) be rerouted over another logging backend.

My all-time-favourite is still Apache Commons Logging, since it is very easy to re-route, widely used, and very stable (no releases since ages). I usually re-route it to log4j using SLF4J, but that's only an option, not a must.

[AngledLuffa]

Stuff that gets done around here tends to fall into one or more of the
following categories: stuff that interests us, stuff that advances us
academically, and stuff that gets us paid. In all honesty, switching the
logging system from System.err.println() to anything else doesn't fall into
any of those three categories. It's always possible that changes in the
future, but for now, I don't expect us to put effort into this.

It does sound like if we were to use any alternate framework, Java Utility
Logging would be the winner, since we wouldn't have to depend on any
outside libraries to use it. If you or anyone else is interested in
contributing this on their own, please keep that in mind.

On Fri, Sep 13, 2013 at 12:05 AM, reckart notifications@github.com wrote:

IMHO log4j would be a pretty bad choice, since it is a logging framework,
not a logging API. Apache Commons Logging for example is split into an API
and an implementation part. The default implementation can be replaced with
e.g. a SLF4J bridge which can then log using any logging backend, including
log4j. There is also a logging API built directly into Java: Java Utility
Logging - it requires no extra dependencies and can likewise (although with
a bit of extra effort) be rerouted over another logging backend. My
all-time-favourite is still Apache Commons Logging, since it is very easy
to re-route, widely used, and very stable (no releases since ages).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1#issuecomment-24376874
.

[manning]

On writing to stdout/stderr: There are certainly a lot of things written to stderr. However, there should not be any logging/error messages written to stdout. If anyone finds any, we will fix them. Things should (and at least mainly do) run cleanly so that only genuine NLP output gets sent to stdout.

Logging: There is now some use of logging in CoreNLP (whereas a few years ago, there was none). I think my preference would be to stick to j.u.l, which is what is used a bit now in the code, just because the marginal gains in flexibility from some of the other frameworks just don't seem to outweigh the cost of adding another library dependency (exacerbated by the fact that there are several different ones that different people like...).

Our use of logging is likely to increase gradually over time, but, as AngledLuffa commented, there's not likely to be someone tasked to convert everything. If someone like you wanted to take this on, we'd certainly be willing to support a change to j.u.l everywhere.

[dinoboy197]

I'd be very interested in getting involved here. I think having CoreNLP write to a logging API which can route into any particular logging backend of the library user's choice seems like the right approach - this seems pretty industry standard at this point. It's unfortunate to write / maintain / use programs which maintain a nice separation between logging API and logging backend for the purpose of maintaining consistent logging format, level, and log destination, only to include libraries which don't take advantage of these nice logging APIs and just dump unformatted text onto stdout / stderr (I don't mean to pick on CoreNLP here alone - there are hundreds, probably thousands of libraries in Maven Central which blindly dump text onto stdout / stderr).

As for a logging API for CoreNLP to interface with, my recommendation would be to use SLF4J. It doesn't bias the client code (which makes calls to CoreNLP) to use any particular logging backend, nor does it require client code to provide additional backend shims to re-route logging events between backends. For example, by using SLF4j as the logging API, a program including the CoreNLP library needs only to include a single logging backend of their choice, for example logback (the successor to log4j), log4j, commons-logging, or java logging (jul). No logging bridges or shims are required for the end client to use any logging backend that they choose. (CoreNLP -> SLF4j API -> SLF4j implementation -> logging backend -> log destination)

If CoreNLP used commons-logging as the logging API, then programs which want to use logback, log4j, or jul as the backend would need to include some sort of log event redirection shim (as reckart mentioned above) (typically in the form of SLF4j plus some additional commons-logging to destination backend bridge) which increases logging backend complexity, adds additional library dependencies to the client program, and increases logging event overhead. (CoreNLP -> commons-logging -> slf4j-commons-logging-shim -> slf4j API -> slf4j implementation -> logging backend -> log destination).

If CoreNLP used JUL as the logging API, then programs which want to use logback, log4j, or commons-logging as the backend need the shims described in the paragraph above - plus, JUL + shims has some very unfortunate performance penalties as desribed (look for jul-to-slf4j bridge "Note on performance" section). (CoreNLP -> jul -> slf4j-jul-shim -> slf4j API -> slf4j implementation -> logging backend -> log destination).

Anyway, I'd love to discuss further and/or get involved to help out.

[reckart]

The thing is the Commons Logging is basically done - I doubt it will ever change again - as stable as it can be. While with SLF4J I've run into trouble in the past due to conflicting versions of SLF4J being on the classpath, etc. I like the SFL4J and I think it's a great choice for applications - for frameworks, I'd still vote for commons logging. If you want to use commons logging with log4j, no additional shims are needed. Just add log4j to the classpath and commons logging picks it up and uses it automatically. If you prefer another backend, you can route it via SLF4J.

[dinoboy197]

Cool, thanks for the insight. If CoreNLP is starting to migrate to Commons Logging, then that's fine - maybe I could help to just finish it up? The latest stable release still writes to stderr without Commons Logging, in places.

Question for Commons Logging: do you happen to know if a library (like CoreNLP) uses Commons Logging as the API and client code uses Logback (which is log4j's "successor") as the backend, does that work without additional shims (ie, no need to SLF4j or any other libraries to be on the classpath?)

[reckart]

I don't think the folks in Stanford are convinced just yet that they want to do this, but maybe if somebody comes up with a patch? No idea... I guess it will be quite some work and would be a pity if they later didn't merge it.

Unless logback comes with a "shim" that allows it to pose as another framework such as log4j, I doubt it. ACL apparently supports these frameworks: Log4j (1.2), LogKit, Avalog, and Java 1.4.

Btw. I just noted that ACL indeed is still being improved - veeery slowly - and activity picked up only recently again: there were 6 years between the release of 1.1.1 and 1.1.2. The latest version is 1.2 (Jul 2014). Changes are apparently minimal and mostly bugfixes.

http://commons.apache.org/proper/commons-logging/changes-report.html

[markwoon]

@dinoboy197 You'll need to use SLF4J if you want to use/control/configure both logback and ACL.

[dinoboy197]

Btw. I just noted that ACL indeed is still being improved - veeery slowly - and activity picked up only recently again: there were 6 years between the release of 1.1.1 and 1.1.2. The latest version is 1.2 (Jul 2014). Changes are apparently minimal and mostly bugfixes.

Heh. Better late than never, right? :)

Anyway, as crazy as it may seem, I may write a patch to convert library methods to use ACL versus println and see if the CoreNLP folks are interested. Thanks for your feedback, @reckart!

[manning]

Hi everyone, good news: We are convinced to start moving to logging! Our plan is to go with slf4j, essentially for the reasons @dinoboy197 gives: It just seems the most modern, most flexible facade, and it seems to be winning in practice: http://zeroturnaround.com/rebellabs/the-state-of-logging-in-java-2013/ . This is likely to be a slow, gradual implementation for the reasons @AngledLuffa gave, but in the next release, you should start to see slf4j appearing and j.u.l -- and a few System.err.println's -- disappearing. Happy to have help. We are actually at 0 unmerged pulled requests right now (!).

[Hronom]

Any progress in moving to logging?
Print in error stream message that not an error not good aproach, for example edu.stanford.nlp.util.Timing#startDoing:1500.
Current version: 3.5.2

Or at least give to user an ability to put custom LogListener

[dinoboy197]

Hi everyone, good news: We are convinced to start moving to logging! Our plan is to go with slf4j, essentially for the reasons @dinoboy197 gives

Super awesome! I missed this message about a month ago. Very excited!

Happy to have help.

@manning - What, in particular can I do to help out this effort? Once slf4j-api is integrated into the build, I can start replacing print statements with slf4j statements, but am not sure if you want smaller PRs, larger PRs, PRs for certain classes, packages, etc. Let me know!