Lost files in the slimmed image for nginx fat image

[isula-xufeng]

Hi,

I run docker-slim to generated a slimmed nginx images:

$ ./docker-slim build --http-probe centos:nginx

And the command generated a 6.4MB image successfully, however, I can't run this slimmed image due to missed file:

$ docker run centos.slim
   2018/02/23 07:52:36 [emerg] 1#1: open() "/etc/nginx/nginx.conf" failed (2: No such file or directory)

I checked artifacts reports, and find nginx.conf file is recorded in creport.json:
"/etc/nginx/nginx.conf": {
"event_count": 2,
"first_eid": 0,
"reads": 1
},
But check "files" directory in artifacts, I can't find nginx.conf:

$ cd files/etc/
$ ll
total 4
-rw-r--r--. 1 root root 590 Nov  8  2016 group

I don't know what's wrong with docker-slim tool.

[kcq]

Thank you for reporting the issue! I really appreciate it. I'd like to reproduce the problem. Can you tell me more about the Docker image. Where is it from? What's in it? What ports is it supposed to expose?

[isula-xufeng]

Sure, thanks for your reply!

This docker image is come from our private repo, so I don't know where is it from, but I guess the official nginx image in docker hub could probably reproduce this issue.
Here is is the Dockerfile.fat generated for your reference:

ADD file:41ea5187c50116884c38d9ec51d920d79cfaeb2a61c52e07a97f457419a10a4f in /
 CMD ["/bin/bash"]
 MAINTAINER NGINX Docker Maintainers "docker-maint@nginx.com"
 ENV NGINX_VERSION=1.11.5-1~jessie
RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 && \
        echo "deb http://nginx.org/packages/mainline/debian/ jessie nginx" >> /etc/apt/sources.list && \
        apt-get update && \
        apt-get install --no-install-recommends --no-install-suggests -y                                                ca-certificates                                                 nginx=${NGINX_VERSION}                                          nginx-module-xslt                                               nginx-module-geoip                                              nginx-module-image-filter                                               nginx-module-perl                                               nginx-module-njs                                                gettext-base && \
        rm -rf /var/lib/apt/lists/*
RUN ln -sf /dev/stdout /var/log/nginx/access.log && \
        ln -sf /dev/stderr /var/log/nginx/error.log
 EXPOSE 443/tcp 80/tcp
 CMD ["nginx" "-g" "daemon off;"]

[kcq]

Thanks for the additional context! Are your web assets packaged in that nginx Docker image? Is it a stand-alone container or is it supposed to be used with other containers (e.g., a data container)? Can you give examples of how you normally use this Docker images? What are the command line parameters to 'docker run'? Do you need to mount any volumes or is it always like the example you provided in the original report?

Thanks again!

[isula-xufeng]

After debug, I found this problem happens in below codes:

internal/app/sensor/data_porcessor.go: 73
func findSymlinks(files []string, mp string) map[string]*report.ArtifactProps {
    result := make(map[string]*report.ArtifactProps, 0)

    //getting the root device is a leftover from the legacy code (not really necessary anymore)
    devID, err := getFileDevice(mp)
    if err != nil {
        return result
    }

    log.Debugf("findSymlinks - deviceId=%v", devID)

    inodes, devices := filesToInodesNative(files)
    inodeToFiles := make(map[uint64][]string)

    //native filepath.Walk is a bit slow (compared to the "find" command)
    //but it's fast enough for now
    filepath.Walk(mp, func(fullName string, fileInfo os.FileInfo, err error) error {
        log.Infof("findSymlinks: process file: %s", fullName)                                 <====== print every files or directories
        sysStatInfo, ok := fileInfo.Sys().(*syscall.Stat_t)
        if !ok {
            return fmt.Errorf("findSymlinks - could not convert fileInfo to Stat_t for %s", fullName)
        }

        if _, ok := devices[uint64(sysStatInfo.Dev)]; !ok {
            return filepath.SkipDir
        }

        if fileInfo.Mode()&os.ModeSymlink != 0 {
            if info, err := getFileSysStats(fullName); err == nil {

                if _, ok := inodes[info.Ino]; ok {
                    //not using the inode for the link (using the target inode instead)
                    inodeToFiles[info.Ino] = append(inodeToFiles[info.Ino], fullName)
                } else {
                    //log.Debugf("findSymlinks - don't care about this symlink (%s)",fullName)
                }

            } else {
                log.Infof("findSymlinks - could not get target stats info for %v", fullName)
            }

        } else {
            if _, ok := inodes[sysStatInfo.Ino]; ok {
                inodeToFiles[sysStatInfo.Ino] = append(inodeToFiles[sysStatInfo.Ino], fullName)
            } else {
                //log.Debugf("findSymlinks - don't care about this file (%s)",fullName)
            }
        }

        return nil
    })

    for inodeID := range inodes {
        v := inodeToFiles[inodeID]
        for _, f := range v {
            //result[f] = inodeID
            result[f] = nil
        }
    }

    return result
}

These codes traverse the container root filesystem, but I found filepath.Walk() doesn't walk some files or directories(I print every walked files or directories).
Not for sure if this is a filepath.Walk() function problem.
BTW, I'm using go version: 1.8.3

[isula-xufeng]

I also tried findSymlinksLegacy() API which use "find" tool instead of findSymlinks(filepath.Walk())，looks like it doesn't have the same problem, but findSymlinksLegacy() has one other problem: it can't detect the symbolic link, and this makes the slimmed images much bigger.

Moreover, this problem is not limited to nginx container, the apache httpd container shares the same problem, and I think much more.

[isula-xufeng]

@kcq
I have confirmed, this is really a bug of docker-slim.
findSymlinks() should determine whether or not the device file is a directory before SkipDir, Otherwise, all the left files or directories will be skipped.

The fix can be like this:

--- a/internal/app/sensor/data_porcessor.go
+++ b/internal/app/sensor/data_porcessor.go
@@ -93,7 +93,11 @@ func findSymlinks(files []string, mp string) map[string]*report.ArtifactProps {
        }

        if _, ok := devices[uint64(sysStatInfo.Dev)]; !ok {
-           return filepath.SkipDir
+           if fileInfo.Mode().IsDir() {
+               return filepath.SkipDir
+           } else {
+               return nil
+           }
        }

        if fileInfo.Mode()&os.ModeSymlink != 0 {

[kcq]

@isula-xufeng The new 1.24 release includes a number of symlink related enhancements address this issue ( https://github.com/docker-slim/docker-slim/releases/tag/1.24 ). Can you try the new version to see if it works for you? Thanks a lot for providing the code snippets!

[kcq]

Closing it for now... Please reopen if you still see the same problem.