projectatomic/docker forked from moby/moby

Fixes case where shutdown occurs before content is synced to disked
on layer creation. This case can leave the layer store in an bad
state and require manual recovery. This change ensures all files
are synced to disk before a layer is committed. Any shutdown that
occurs will only cause the layer to not show up but will allow it to
be repulled or recreated without error.

Added generic io logic to ioutils package to abstract it out of
the layer store package.


Signed-off-by: Derek McGowan <derek@mcgstyle.net>

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

Attempt to avoid leaking or holding open descriptors for deleted journal files

Fix moby#31350

As we can see in `daemon.Images()`, there is a gap between
`allImages = daemon.imageStore.Map()` and `l, err :=
daemon.layerStore.Get(layerID)`, so images which still exist when we hit
`allImages = daemon.imageStore.Map()` may have already been deleted when we hit
`l, err := daemon.layerStore.Get(layerID)`.

```
	if danglingOnly {
        	allImages = daemon.imageStore.Heads()
	} else {
        	allImages = daemon.imageStore.Map()
	}

	...

	for id, img := range allImages {
		...

		layerID := img.RootFS.ChainID()
        	var size int64
	        if layerID != "" {
        	        l, err := daemon.layerStore.Get(layerID)
                	if err != nil {
                        	return nil, err
                	}
```

Signed-off-by: Yuanhong Peng <pengyuanhong@huawei.com>

Sanitize docker labels when used as journald field names

This fix tries to address the issue raised in moby#23528 where
docker labels caused journald log error because journald
has special requirements on field names.

This fix addresses this issue by sanitize the labels per
requirements of journald.

Additional unit tests have been added to cover the changes.

This fix fixes moby#23528.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

Adjust Audit init in 1.12.6 to match 1.13.1

Log an error if sd_journal_previous() returns an error code.  If it
returns 0, keep accepting that as an indication that we've hit the
beginning of the journal, and that there's nothing to be done.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

Fix a comment.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

If the output channel gets stalled while reading the journal in
non-follow mode, don't stop right away.  Doing this correctly means that
we need to detect when we've hit the end of the log, so start returning
a value indicating whether or not we ran out of entries to read.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

When reading from the journal, handle seeking to the read location
outside of the main reading loop.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

Clean up a deferred function call in the journal reading logic.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

When checking for updates to the journal, treat INVALIDATE results like
we do APPEND results, because the library can't signal both, and in
either case we need to figure out what's going on.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

Periodically call sd_journal_process() when reading the journal, to skip
over journal files which have been deleted since we started reading the
journal, for cases where our keeping them open contributes to a shortage
of disk space.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

The journal API opens the journal files when the journal handle is
opened by sd_journal_open(), and uses an inotify descriptor to notice
when files have been removed, but it doesn't set up that descriptor
until the first time that sd_journal_get_fd() is called (either by a
client, or as part of sd_journal_wait()).

We hadn't been doing that until our initial read-through of entries was
done, meaning that we've been missing file deletion events that occurred
while we were reading the journal.  Make that window shorter.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

When draining the journal, if the output message channel is full, stop
reading.  In non-follow mode, keep reading until we start seeing entries
that post-date the time when we started reading the journal.  Otherwise,
if we can't keep up with the journal, we never actually stop reading.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

Fix https://bugzilla.redhat.com/show_bug.cgi?id=1540540

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>

Initialize Audit loggings cli.daemon

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>

Upstream reference:
moby@62871ef
Fix bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1526369

Closes: moby#33412

Signed-off-by: Felix Abecassis <fabecassis@nvidia.com>
Signed-off-by: Antonio Murdaca <runcom@redhat.com>

BACKPORT: devicemapper: remove container rootfs mountPath after umount

Upstream commit: 92e45b8

libdm currently has a fairly substantial DoS bug that makes certain
operations fail on a libdm device if the device has active references
through mountpoints. This is a significant problem with the advent of
mount namespaces and MS_PRIVATE, and can cause certain --volume mounts
to cause libdm to no longer be able to remove containers:

  % docker run -d --name testA busybox top
  % docker run -d --name testB -v /var/lib/docker:/docker busybox top
  % docker rm -f testA
  [fails on libdm with dm_task_run errors.]

This also solves the problem of unprivileged users being able to DoS
docker by using unprivileged mount namespaces to preseve mounts that
Docker has dropped.

Signed-off-by: Aleksa Sarai <asarai@suse.de>

Upstream reference: moby#35732

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

This is writeable, and can be used to remove devices. Containers do
not need to know about scsi devices.

Fix https://nvd.nist.gov/vuln/detail/CVE-2017-16539

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

… to 1.14-dev

Upstream reference: docker/libnetwork#1620
Fix https://bugzilla.redhat.com/show_bug.cgi?id=1464790

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

…daemon

Fix https://bugzilla.redhat.com/show_bug.cgi?id=1510170

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

Fix https://bugzilla.redhat.com/show_bug.cgi?id=1511442

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

Signed-off-by: Antonio Murdaca <runcom@redhat.com>

Upstream referece:
moby@72df48d
Fix Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1498553

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Antonio Murdaca <runcom@redhat.com>

/dev should not be readonly with --readonly flag

/dev is mounted on a tmpfs inside of a container.  Processes inside of containers
some times need to create devices nodes, or to setup a socket that listens on /dev/log
Allowing these containers to run with the --readonly flag makes sense.  Making a tmpfs
readonly does not add any security to the container, since there is plenty of places
where the container can write tmpfs content.

I have no idea why /dev was excluded.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Put start/exec/create/remove events in debug to lower log spam