New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[17.06] Fix log readers can block writes indefinitely #98

Merged
merged 2 commits into from Jul 12, 2017

Conversation

Projects
None yet
2 participants
@andrewhsu
Collaborator

andrewhsu commented Jul 6, 2017

Backport fix:

The only conflict was in the components/engine/daemon/logger/jsonfilelog/read.go file:

  if config.Tail != 0 {
<<<<<<< HEAD
    tailer := ioutils.MultiReadSeeker(append(files, latestFile)...)
=======
    tailer := multireader.MultiReadSeeker(append(files, latestChunk)...)
>>>>>>> e2209185ed... Fix log readers can block writes indefinitely
    tailFile(tailer, logWatcher, config.Tail, config.Since)
  }

@cpuguy83 Because moby/moby@2445e6b moved multireader. I decided to go with the ioutils line to resolve the conflict. Let me know if I should go the other way and and also cherry-pick the multireader move commit.

Fix log readers can block writes indefinitely
Before this patch, a log reader is able to block all log writes
indefinitely (and other operations) by simply opening the log stream and
not consuming all the messages.

The reason for this is we protect the read stream from corruption by
ensuring there are no new writes while the log stream is consumed (and
caught up with the live entries).

We can get around this issue because log files are append only, so we
can limit reads to only the section of the file that was written to when
the log stream was first requested.

Now logs are only blocked until all files are opened, rather than
streamed to the client.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e220918)

Conflicts:
components/engine/daemon/logger/jsonfilelog/read.go
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jul 6, 2017

Contributor

22:08:13 [init] daemon/logger/jsonfilelog/read.go:80: latestChunk declared and not used

Contributor

cpuguy83 commented Jul 6, 2017

22:08:13 [init] daemon/logger/jsonfilelog/read.go:80: latestChunk declared and not used

use latestChunk instead of latestFile to get cherry-pick commit 7bd7bde
… working

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
@cpuguy83

LGTM

@andrewhsu andrewhsu merged commit 0aba544 into docker:17.06 Jul 12, 2017

3 checks passed

ce-tests Jenkins build docker-ce-17.06-pr 147 has succeeded
Details
ce-tests-WoW-RS1 Jenkins build docker-ce-17.06-pr-WoW-RS1 135 has succeeded
Details
dco-signed All commits are signed

@andrewhsu andrewhsu deleted the andrewhsu:fix-log-readers branch Jul 12, 2017

@andrewhsu andrewhsu modified the milestone: 17.06.1 Jul 12, 2017

docker-jenkins pushed a commit that referenced this pull request Apr 4, 2018

Merge pull request #98 from seemethere/f28
Add packaging code for Fedora 28
Upstream-commit: 9efc79dd124c6c2a07488e68ddbad1558b8e5e30
Component: packaging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment