Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffer lock can block client (i.e, dockerd) #54

Closed
stefano-pogliani opened this issue Sep 13, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@stefano-pogliani
Copy link

commented Sep 13, 2017

Hello,

We have experienced a couple of blocked web server containers in our production infrastructure caused by fluentd being overloaded.

I suspect this is caused by the attempt of send to flush the entire buffer and keeping a hold on the buffer lock while doing so.
If the fluend server is slow for any reason the send method will slowly loop until the buffer is empty.
In the meantime, any attempt to log will block at the appendBuffer level because send still has the lock.

While looking for known issues/prs I found #33 and the implementing PR from @alexlry #52 which would remove the lock altogether.

It would be great if that PR could be reviewed and merged in so that docker containers would not longer block while trying to write to stdout but rather fail with full buffers.

I have also built a Gist detailing how to reproduce this issue with docker-compose and tc: https://gist.github.com/stefano-pogliani/463af63c040d1bd3740fb46255d2768a

@alexlry alexlry referenced this issue Apr 9, 2018

Merged

Async send #56

@thaJeztah

This comment has been minimized.

Copy link

commented Apr 16, 2019

@tagomoris @alexlry should this one be closed (resolved by #56)?

@tagomoris

This comment has been minimized.

Copy link
Member

commented Apr 16, 2019

@thaJeztah Indeed.

@tagomoris tagomoris closed this Apr 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.