New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add top level support for logging #130

Open
savaki opened this Issue Jun 30, 2018 · 6 comments

Comments

Projects
None yet
4 participants
@savaki
Copy link

savaki commented Jun 30, 2018

In addition to metrics and traces, opencensus should also include support for logging. Tags are an incredibly useful feature and being able to share a common set of tags across metrics and logging is incredibly useful not to mention the ability to correlate logs with traces as referenced here:

#123

@adriancole

This comment has been minimized.

Copy link

adriancole commented Jun 30, 2018

@songy23

This comment has been minimized.

Copy link
Member

songy23 commented Jul 2, 2018

@savaki

This comment has been minimized.

Copy link
Author

savaki commented Jul 16, 2018

Using go as a frame of reference, what I'd love to see is a trace and tag aware logging interface that can be implemented by different providers.

We use opencensus-go internally and have written an internal library to handle this that we've found to be exceptionally useful. Being able to filter log records by a particular user or org make troubleshooting much easier. This is even more magical when coupled with the log tailing feature of Stackdriver.

To move the conversation forward, I put together a working experiment of what this library might look like, https://github.com/savaki/opencensus-go/tree/logging/log Like sl4j, this log package is an interface without an implementation. In go, I'd expect exporters would be written for logrus, zap, and other popular packages

In languages like Java, I would imagine we would want to make available exporters for common packages like sl4j or log4j

@adriancole

This comment has been minimized.

Copy link

adriancole commented Jul 16, 2018

Thanks for the info. so this sounds like a primary interface for logging, then. Ex user code would use it directly instead of another library. Would you expect this only to be the case in languages where there is no dominant logging library? Ex in java would you expect folks to use something like this instead of SLF4J or log4j2 etc? Or would you use an integration pattern there instead

(asking the more general question as this is a spec repo so seemingly cross language)

@savaki

This comment has been minimized.

Copy link
Author

savaki commented Jul 17, 2018

That's a good question.

As I understand it, one of the main value propositions of OpenCensus is that it provides standard interface for common operational operations like tracing and metrics across many languages and many service providers. When I write to the OpenCensus API, I have the promise of portability between the supported providers. It's reasonably easy to swap out Zipkin for Stackdriver for example.

The intent of this proposal is to extend that promise to logging. Disclaimer: I'm not very familiar with the OC Java implementation. That said, it seems that if we want to allow users to integrate logging an existing Java app with OC, there are a number of approaches.

Configuration Only
In this approach, no code changes are required. All the magic would be done inside the SLF4J or log4j2 configuration using a custom appender like thing. I'm a little confused as to how this would work as I understand propagating ThreadLocal data over futures is still an outstanding issue. And if developers haven't changed their code and they don't have access to ThreadLocal data, I'm not sure how OC would get trace or tag data. Granted you could give a big disclaimer that this approach only works with non-async code, but that doesn't seem like a good approach.

Custom Marker
In the case of SLF4J, it looks like the have this Marker thing that might be appropriated for OpenCensus use. I haven't used it myself so I'm only guessing based on the apidoc. In this case, where ever we want the addition trace/tag data, we'd need the developer to inject an OC aware Marker. And given there may be existing code, it seems reasonable to expect OC to have to support wrapping existing Markers. I also need extend SLF4J to support OC configurations so swapping providers becomes a little more difficult. And while this seems like it ties OC to SLF4J, given the wide adoption of these libraries, for Java, I understand something like this might be the way to go.

Primary Interface
However, if the developer already has to modify the code where ever they would like to get this augmented data, why not provide a common interface and have them write to that? The would be no messy interaction between OC and existing marker like types. Swapping providers would be easier. The common interface could still write out to log4j.

I'm not a Java expert, so I certainly admit I could be missing something.

@adriancole

This comment has been minimized.

Copy link

adriancole commented Jul 17, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment