Scan docker image config metadata

[AlfredBerg]

Description:

This is my attempt at solving #3007
The current implementation only scans the created_by value in the docker image config (https://github.com/opencontainers/image-spec/blob/main/config.md). This misses e.g. the environment variables of the docker image and other fields that commonly contain secrets.

The easiest way to view this metadata is by using "crane config [image]" https://github.com/google/go-containerregistry/tree/main/cmd/crane

Checklist:

• Tests passing (make test-community)? (fails but also fails on main without my changes)
• Lint passing (make lint this requires golangci-lint)?

[CLAassistant]

All committers have signed the CLA.

[AlfredBerg]

I don't really know if it is worth keeping this metric as in most cases it will be the same as dockerImagesScanned

[rosecodym]

Hey, sorry for letting this fall through the cracks. I'm no docker expert - does this PR preserve the existing behavior of scanning each history entry's createdBy?

[AlfredBerg]

Hey, sorry for letting this fall through the cracks. I'm no docker expert - does this PR preserve the existing behavior of scanning each history entry's createdBy?

Yes, it scans the whole config file that contains the createdBy entry's

Here is an example of what the nginx conf looks like from crane config nginx | jq, note the "created_by" fields

Nginx docker conf

{
  "architecture": "amd64",
  "config": {
    "ExposedPorts": {
      "80/tcp": {}
    },
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "NGINX_VERSION=1.27.4",
      "NJS_VERSION=0.8.9",
      "NJS_RELEASE=1~bookworm",
      "PKG_RELEASE=1~bookworm",
      "DYNPKG_RELEASE=1~bookworm"
    ],
    "Entrypoint": [
      "/docker-entrypoint.sh"
    ],
    "Cmd": [
      "nginx",
      "-g",
      "daemon off;"
    ],
    "Labels": {
      "maintainer": "NGINX Docker Maintainers <docker-maint@nginx.com>"
    },
    "StopSignal": "SIGQUIT"
  },
  "created": "2025-02-05T21:27:16Z",
  "history": [
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "# debian.sh --arch 'amd64' out/ 'bookworm' '@1740355200'",
      "comment": "debuerreotype 0.15"
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "LABEL maintainer=NGINX Docker Maintainers <docker-maint@nginx.com>",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "ENV NGINX_VERSION=1.27.4",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "ENV NJS_VERSION=0.8.9",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "ENV NJS_RELEASE=1~bookworm",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "ENV PKG_RELEASE=1~bookworm",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "ENV DYNPKG_RELEASE=1~bookworm",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "RUN /bin/sh -c set -x     && groupadd --system --gid 101 nginx     && useradd --system --gid nginx --no-create-home --home /nonexistent --comment \"nginx user\" --shell /bin/false --uid 101 nginx     && apt-get update     && apt-get install --no-install-recommends --no-install-suggests -y gnupg1 ca-certificates     &&     NGINX_GPGKEYS=\"573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 8540A6F18833A80E9C1653A42FD21310B49F6B46 9E9BE90EACBCDE69FE9B204CBCDCD8A38D88A2B3\";     NGINX_GPGKEY_PATH=/etc/apt/keyrings/nginx-archive-keyring.gpg;     export GNUPGHOME=\"$(mktemp -d)\";     found='';     for NGINX_GPGKEY in $NGINX_GPGKEYS; do     for server in         hkp://keyserver.ubuntu.com:80         pgp.mit.edu     ; do         echo \"Fetching GPG key $NGINX_GPGKEY from $server\";         gpg1 --keyserver \"$server\" --keyserver-options timeout=10 --recv-keys \"$NGINX_GPGKEY\" && found=yes && break;     done;     test -z \"$found\" && echo >&2 \"error: failed to fetch GPG key $NGINX_GPGKEY\" && exit 1;     done;     gpg1 --export \"$NGINX_GPGKEYS\" > \"$NGINX_GPGKEY_PATH\" ;     rm -rf \"$GNUPGHOME\";     apt-get remove --purge --auto-remove -y gnupg1 && rm -rf /var/lib/apt/lists/*     && dpkgArch=\"$(dpkg --print-architecture)\"     && nginxPackages=\"         nginx=${NGINX_VERSION}-${PKG_RELEASE}         nginx-module-xslt=${NGINX_VERSION}-${DYNPKG_RELEASE}         nginx-module-geoip=${NGINX_VERSION}-${DYNPKG_RELEASE}         nginx-module-image-filter=${NGINX_VERSION}-${DYNPKG_RELEASE}         nginx-module-njs=${NGINX_VERSION}+${NJS_VERSION}-${NJS_RELEASE}     \"     && case \"$dpkgArch\" in         amd64|arm64)             echo \"deb [signed-by=$NGINX_GPGKEY_PATH] https://nginx.org/packages/mainline/debian/ bookworm nginx\" >> /etc/apt/sources.list.d/nginx.list             && apt-get update             ;;         *)             tempDir=\"$(mktemp -d)\"             && chmod 777 \"$tempDir\"                         && savedAptMark=\"$(apt-mark showmanual)\"                         && apt-get update             && apt-get install --no-install-recommends --no-install-suggests -y                 curl                 devscripts                 equivs                 git                 libxml2-utils                 lsb-release                 xsltproc             && (                 cd \"$tempDir\"                 && REVISION=\"${NGINX_VERSION}-${PKG_RELEASE}\"                 && REVISION=${REVISION%~*}                 && curl -f -L -O https://github.com/nginx/pkg-oss/archive/${REVISION}.tar.gz                 && PKGOSSCHECKSUM=\"973690e64fa47e3704e817a3b08205b9e3f8c0cbe31825d9d62a81c11eb3aa186df015f27fdfd48c8799ffc528e38a9168c592ae665e4835c2d28638ec5f7845 *${REVISION}.tar.gz\"                 && if [ \"$(openssl sha512 -r ${REVISION}.tar.gz)\" = \"$PKGOSSCHECKSUM\" ]; then                     echo \"pkg-oss tarball checksum verification succeeded!\";                 else                     echo \"pkg-oss tarball checksum verification failed!\";                     exit 1;                 fi                 && tar xzvf ${REVISION}.tar.gz                 && cd pkg-oss-${REVISION}                 && cd debian                 && for target in base module-geoip module-image-filter module-njs module-xslt; do                     make rules-$target;                     mk-build-deps --install --tool=\"apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes\"                         debuild-$target/nginx-$NGINX_VERSION/debian/control;                 done                 && make base module-geoip module-image-filter module-njs module-xslt             )                         && apt-mark showmanual | xargs apt-mark auto > /dev/null             && { [ -z \"$savedAptMark\" ] || apt-mark manual $savedAptMark; }                         && ls -lAFh \"$tempDir\"             && ( cd \"$tempDir\" && dpkg-scanpackages . > Packages )             && grep '^Package: ' \"$tempDir/Packages\"             && echo \"deb [ trusted=yes ] file://$tempDir ./\" > /etc/apt/sources.list.d/temp.list             && apt-get -o Acquire::GzipIndexes=false update             ;;     esac         && apt-get install --no-install-recommends --no-install-suggests -y                         $nginxPackages                         gettext-base                         curl     && apt-get remove --purge --auto-remove -y && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx.list         && if [ -n \"$tempDir\" ]; then         apt-get purge -y --auto-remove         && rm -rf \"$tempDir\" /etc/apt/sources.list.d/temp.list;     fi     && ln -sf /dev/stdout /var/log/nginx/access.log     && ln -sf /dev/stderr /var/log/nginx/error.log     && mkdir /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "COPY docker-entrypoint.sh / # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "COPY 10-listen-on-ipv6-by-default.sh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "COPY 15-local-resolvers.envsh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "COPY 20-envsubst-on-templates.sh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "COPY 30-tune-worker-processes.sh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "ENTRYPOINT [\"/docker-entrypoint.sh\"]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "EXPOSE map[80/tcp:{}]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "STOPSIGNAL SIGQUIT",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2025-02-05T21:27:16Z",
      "created_by": "CMD [\"nginx\" \"-g\" \"daemon off;\"]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:5f1ee22ffb5e68686db3dcb6584eb1c73b5570615b0f14fabb070b96117e351d",
      "sha256:c68632c455ae0c46d1380033bae6d30014853fa3f600f4e14efc440be1bc9580",
      "sha256:cabea05c000e49f0814b2611cbc66c2787f609d8a27fc7b9e97b5dab5d8502da",
      "sha256:791f0a07985c2814a899cb0458802be06ba124a364f7e5a9413a1f08fdbf5b5c",
      "sha256:f6d5815f290ee912fd4a768d97b46af39523dff584d786f5c0f7e9bdb7fad537",
      "sha256:7d22e2347c1217a89bd3c79ca9adb4652c1e9b61427fffc0ab92227aacd19a38",
      "sha256:55e9644f21c38d7707b4a432aacc7817c5414b68ac7a750e704c2f7100ebc15c"
    ]
  }
}

[rosecodym]

Ok, thanks! I do see that you've removed the Layer metadata entry from these secrets. That makes total sense, but unfortunately, removing metadata is something we try really hard to avoid because anyone using TruffleHog as part of a workflow that persistently tracks secrets will see metadata change, which can cause very irritating problems. This unfortunately isn't esoteric - it's every user of TruffleHog Enterprise 😞

How much work would it be to somehow retrieve layer information for each of these found secrets so that that metadata field doesn't change?

[rosecodym]

Also, we should use a ChunkReader to scan the config file in case it gets really big. (The docker source already does this for the layers themselves.)

[AlfredBerg]

Ok, thanks! I do see that you've removed the Layer metadata entry from these secrets. That makes total sense, but unfortunately, removing metadata is something we try really hard to avoid because anyone using TruffleHog as part of a workflow that persistently tracks secrets will see metadata change, which can cause very irritating problems. This unfortunately isn't esoteric - it's every user of TruffleHog Enterprise 😞

How much work would it be to somehow retrieve layer information for each of these found secrets so that that metadata field doesn't change?

I understand, I'll see if I can come up with something, though I will probably have to parse the json file then. However will be a few weeks before I can look into it