Add a syslog encoder and sink

[lysu]

now zap can write to stderr and stdout...

Is any plan to support syslog?

[akshayjshah]

Eventually, yes - supporting syslog will require both a syslog encoder (similar to zap's current JSON encoder) and a WriteSyncer.

We're definitely not going to get to this until after a 1.0 release, though.

[skypacer210]

Hi @akshayjshah
Dose it mean that zap can't write log to file? If zap support write to file, how to use it? Thanks a lot.

[prashantv]

@skypacer210 This issue is not related to writing to a file, but specifically to the syslog interface.

You can write to a file using zap, take a look at zap.Output option. The os.File type satisfies this interface so you can open a file and pass it in as the output.

I've filed #100 so we have an example to point to for file logging.

[skypacer210]

@akshayjshah Thanks a lot!

[skypacer210]

@akshayjshah Do zap support logging with rotation?

[prashantv]

@skypacer210 Please look at existing documentation and issues, there is a separate issue about this: #98. As I mentioned earlier, this issue is not related to writing to a file, but about the syslog interface, so let's move the discussion of logging to file away from this issue.

[skypacer210]

@prashantv Got it, thanks.

[jeanlucmongrain]

I actually implemented a nearly completed Syslog friendly encoder (mostly just a slightly modified JSON encoder) and a syslog writer.

But I can't finish it because zap.Encoder.Encoder interface WriteEntry(io.Writer, string, Level, time.Time) error function first argument io.Writer is not enough.

The problem is: syslog writer need to know the zap.Level and not just the []byte of the encoded log entry. Syslog must specify priority and level on each new log entry.

To achieve this I suggest create the new following interface:

type LogWriter interface {
        Write([]byte, Level) (int, error)
}

and use it instead of io.Writer in WriteEntry.

For writers that don't need the level such to use os.Stdout as a LogWriter, we can use that wrapper:

type wrappedWriter struct {
    w io.Wrapper
}

func (w *wrappedWriter) Write(p []byte, _ Level) (int, error) {
    return w.Write(p)
}

func WrapIgnoreLevel(w io.Wrapper) LogWriter {
    return &wrappedWriter{w}
}

If this change to zap internal API is approved, I can finish implement syslog support.

[prashantv]

Adding Level seems like a change that's very particular to this use-case. I think passing a struct that contains message, time, and level would be cleaner, and this same struct could also be used in the Encoder interface.

This struct is almost the same as Entry, so perhaps we should reduce what Entry is responsible for, and make the following interface changes:

• Entry no longer contains encoder or fields, it would only contain message, level, and time.
• Hook changes from func(*Entry) to func(*Entry, Keyvalue)
• Encoder's WriteEntry changes from WriteEntry(io.Writer, string, Level, time.Time) to WriteEntry(io.Writer, Entry) (or possibly *Entry for efficiency).

[akshayjshah]

@bclermont Thanks for putting in the time to prototype this!

I'd also like to make an interface changes generic enough that we have a fighting chance of supporting other encodings and sinks down the road. Modifying Entry makes sense to me.

However, I would like to preserve the separation of encoders and outputs. While the syslog format and syslog outputs are pretty tightly coupled, this sort of thing is broadly useful - for example, it's reasonable to support different files per log level (or per name, if we implement #130). To support those use cases cleanly, I do think we'll need to rethink the interface that outputs provide; io.Writer doesn't receive enough information to do the job.

@bclermont This discussion might be a bit easier if you open a PR with your code, even if it's incomplete - at least then we'll have a concrete implementation to look at.

[tchap]

In case anybody is interested in syslog, you might want to check https://github.com/tchap/zapext/blob/master/zapsyslog/core.go ;-)

Exemplar usage can be found here: https://github.com/tchap/zapext/blob/master/zapsyslog/example/main.go

[avidak]

I implemented syslog writer in my own application so I made pull request if you are interested in using it in your library

#506

[akshayjshah]

Awesome - thanks, @avidak. Clearly, we've been thinking about this a lot internally without posting updates on this issue - sorry, everyone!

I don't think that it's a good idea to include this functionality inside zap itself. I do agree that it's important, and clearly many of you need it. However, we don't use syslog internally, and none of the zap maintainers have any experience with it. We're committed to supporting this project, including fixing bugs...which will be very difficult for anything syslog-related. As a small example, none of us are certain whether it's okay to send msgpack-encoded (or other binary) logs to syslog.

To make sure that the syslog integration is well-supported and works correctly, it should be maintained by someone who actually uses it and understands syslog. @tchap has already written such an extension, which I'm happy to send users to. I've added a table of extensions to zap's FAQ in #576; as we revamp the zap docs, that table will become more prominent.

Again, sorry for the long silence on this issue. I hope that this approach ends up being good for everyone: users get a supported, battle-tested syslog integration, and zap itself remains understandable to the core maintainers.

I'll leave this issue open for a bit, in case we need a bit more discussion.