performance issue of writing logs

[JimmyMa]

The writerSink needs to acquire a lock before write logs :https://github.com/cloudfoundry/lager/blob/master/writer_sink.go#L34 , if the file IO is slow, that will blocks thousands of goroutines.

Could the writerSink use channel to write the logs, like what gorouter access logs writer does: https://github.com/cloudfoundry/gorouter/blob/master/access_log/file_and_loggregator_access_logger.go#L94

With channel, the logs are cached in memory, and no goroutines are blocked.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/135110101

The labels on this github issue will be updated when the story is started.

[emalm]

Thanks, @JimmyMa. Could you explain more about the concurrency problems you observed with lager? Was the blockage a transient condition that eventually cleared up, or was it a fundamental problem where the log serialization could not keep up with the log message production in the component? I'd like us to understand your exact problem more thoroughly before deciding on a solution.

Best,
Eric, CF Runtime Diego PM

[emalm]

Closing due to inactivity. Please feel free to open a new issue with more detail about the specific problem you've encountered.

Best,
Eric