Scanning Docker images from Google Artifacts Registry from Gitlab CI

[ahsan-raza]

I was using trivy to scan docker images in Gitlab CI job from Gitlab registry. Now I have moved the registry from Gitlab to Google Artifact Registry. I am able to push the image using json auth to Artifact Registry. But whenever I try to implement the scanning of the docker image in Gitlab CI job, it gives me an error of permission denied. I tried to manually run the trivy on Docker for Mac and perform all the steps that I did in my CI job, it works without any problem.

It gives the following error:

Here is my Gitlab CI job:

container_scanning:
  image:
   name: docker.io/aquasec/trivy:latest
   entrypoint: [""]
  allow_failure: false
  only:
    - main
    - develop
    - integration
    - tags

  stage: trivy_scan_build
  variables:
    # No need to clone the repo, we exclusively work on artifacts.  See
    # https://docs.gitlab.com/ee/ci/runners/README.html#git-strategy
    GIT_STRATEGY: none
    # TRIVY_USERNAME: "$CI_REGISTRY_USER"
    # TRIVY_PASSWORD: "$CI_REGISTRY_PASSWORD"
    # TRIVY_AUTH_URL: "$CI_REGISTRY"
    #FULL_IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG

  before_script:
    - mkdir -p ~/.docker && GCR_AUTH=$(echo "_json_key:$GCP_REPO_KEY" | base64 | tr -d '[:space:]')
    - |
      cat << EOF > ~/.docker/config.json
      {"auths":
        {
          "europe-west3-docker.pkg.dev":{"auth":"$GCR_AUTH"}
        }
      }
      EOF
  script:
    - trivy --version
    # cache cleanup is needed when scanning images with the same tags, it does not remove the database
    - trivy image --clear-cache
    # update vulnerabilities db
    - trivy image --download-db-only --no-progress --cache-dir .trivycache/
    # Builds report and puts it in the default workdir $CI_PROJECT_DIR, so `artifacts:` can take it from there
    - trivy image --exit-code 0 --cache-dir .trivycache/ --no-progress --format template --template "@/contrib/gitlab.tpl" --output "$CI_PROJECT_DIR/gl-container-scanning-report.json" "$MICROSERVICE_BRANCH_OR_TAG_IMAGE"
    # Prints full report
    - trivy image --exit-code 0 --cache-dir .trivycache/ --no-progress $MICROSERVICE_BRANCH_OR_TAG_IMAGE > "${CI_PROJECT_TITLE}"-microservice-vulnerabilities-trivy.txt
    # to fail on critical vulnerabilities change to --exit-code 1
    - trivy image --exit-code 0 --cache-dir .trivycache/ --severity CRITICAL --no-progress $MICROSERVICE_BRANCH_OR_TAG_IMAGE   > "${CI_PROJECT_TITLE}"-microservice-vulnerabilities-critical-failed-trivy.txt
  cache:
    paths:
      - .trivycache/
  # Enables https://docs.gitlab.com/ee/user/application_security/container_scanning/ (Container Scanning report is available on GitLab EE Ultimate or GitLab.com Gold)
  artifacts:
    when:                          always
    reports:
      container_scanning:          gl-container-scanning-report.json
    paths:
      - ${CI_PROJECT_TITLE}-microservice-vulnerabilities-trivy.txt
      - ${CI_PROJECT_TITLE}-microservice-vulnerabilities-critical-failed-trivy.txt\

Output of run with -debug:

/trivy image --exit-code 0 --debug --format template --template "@contrib/gitlab.tpl" -o gl-container-scanning-report.json --output "$CI_PROJECT_DIR/gl-container-scanning-report.json" "$MICROSERVICE_BRANCH_OR_TAG_IMAGE"
2022-11-18T22:17:09.127Z	DEBUG	Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2022-11-18T22:17:09.133Z	DEBUG	cache dir:  .trivycache/
2022-11-18T22:17:09.133Z	DEBUG	There is no valid metadata file: unable to open a file: open .trivycache/db/metadata.json: no such file or directory
2022-11-18T22:17:09.133Z	INFO	Need to update DB
2022-11-18T22:17:09.133Z	INFO	DB Repository: ghcr.io/aquasecurity/trivy-db
2022-11-18T22:17:09.133Z	INFO	Downloading DB...
2022-11-18T22:17:09.133Z	DEBUG	no metadata file
2022-11-18T22:17:11.121Z	DEBUG	Updating database metadata...
2022-11-18T22:17:11.122Z	DEBUG	DB Schema: 2, UpdatedAt: 2022-11-18 18:06:59.225394691 +0000 UTC, NextUpdate: 2022-11-19 00:06:59.225394291 +0000 UTC, DownloadedAt: 2022-11-18 22:17:11.121824889 +0000 UTC
2022-11-18T22:17:11.122Z	INFO	Vulnerability scanning is enabled
2022-11-18T22:17:11.122Z	DEBUG	Vulnerability type:  [os library]
2022-11-18T22:17:11.122Z	INFO	Secret scanning is enabled
2022-11-18T22:17:11.122Z	INFO	If your scanning is slow, please try '--security-checks vuln' to disable secret scanning
2022-11-18T22:17:11.122Z	INFO	Please see also https://aquasecurity.github.io/trivy/v0.34/docs/secret/scanning/#recommendation for faster secret detection
2022-11-18T22:17:11.438Z	FATAL	image scan error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.Run
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:370
  - scan error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:230
  - unable to initialize a scanner:
    github.com/aquasecurity/trivy/pkg/commands/artifact.scan
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:549
  - unable to initialize a docker scanner:
    github.com/aquasecurity/trivy/pkg/commands/artifact.imageStandaloneScanner
        /home/runner/work/trivy/trivy/pkg/commands/artifact/scanner.go:22
  - 4 errors occurred:
	* failed to initialize a docker client: unable to parse docker host `localhost`
	* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory
	* containerd socket not found: /run/containerd/containerd.sock
	* GET https://europe-west3-docker.pkg.dev/v2/token?scope=repository%3Aregistry%2Fproject%2Finfrastructure%2Fartifact-registry-test%2Fmicroservice%3Apull&service=: DENIED: Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/registry/locations/europe-west3/repositories/project" (or it may not exist)

Output of trivy -v:

0.34.0

Additional details (base image name, container registry info...):

I tried to run this by downloading trivy inside docker dind in Gitlab CI but the problem is same. Any help is appreciated

[ahsan-raza]

I did some exercise by attaching a docker executor runner with my project on gitlab and it works fine. But the problem occurs only when I am running gitlab-runner in kubernetes executor configurations.

[denisgolius]

Does anybody try to scan images from Github Actions?

[umax]

Hi, @ahsan-raza. What is MICROSERVICE_BRANCH_OR_TAG_IMAGE variable value?

[dpfaffenbauer]

@ahsan-raza I am running into the same issue, did you ever resolve it?

[nestle1936]

Running into same issue too. Any update?

[ahsan-raza]

@umax MICROSERVICE_BRANCH_OR_TAG_IMAGE is the name:tag of the image that is needed to be scanned.
I was not able to solve it. I decided to keep the container scanning on gitlab instead of google artifact registry.

[umax]

@ahsan-raza does just docker pull $MICROSERVICE_BRANCH_OR_TAG_IMAGE work on your GitLab runner?

[ahsan-raza]

yes it does. I build the docker image and push it to Google Artifact Registry on the same Gitlab runner which I use to scan Docker image with trivy.

[ahsan-raza]

I was able to resolve the problem. Here is the solution:

trivy_scan_tag_v2:
  image:
   name: docker.io/aquasec/trivy:0.35.0
   entrypoint: [""]
  stage: scan_image_tag
  variables:
    # No need to clone the repo, we exclusively work on artifacts.  See
    # https://docs.gitlab.com/ee/ci/runners/README.html#git-strategy
    GIT_STRATEGY: none
    TRIVY_USERNAME: ""
    TRIVY_PASSWORD: ""
    TRIVY_AUTH_URL: ""
    GOOGLE_APPLICATION_CREDENTIALS: $GOOGLE_APPLICATION


  script:
    - echo $GOOGLE_APPLICATION_CREDENTIALS > /tmp/service.json
    - export GOOGLE_APPLICATION_CREDENTIALS=/tmp/service.json
    - trivy --version
  cache:
    paths:
      - .trivycache/

while $GOOGLE_APPLICATION is the variable stored in Gitlab CI/CD variables. The value of this variable is the JSON KEY generated by service account of the Google Artifact Registry
Cheers