Verify the checksum

[suzuki-shunsuke]

ref. https://zenn.dev/shunsuke_suzuki/scraps/7cfc2d3a5c6d04

Overview

Verify the checksum of downloaded file, and if the checksum is wrong make the installation failure.

Motivation

Make aqua secure.
Prevent the supply chain atack.

Consideration

We have to keep aqua simple. We should avoid making aqua complicated by introducing checksum verification.

Proposal of Specification

When a tool is installed, aqua verifies the checksum as the following.

1. Download the file in the temporal directory
2. Calculate the checksum from the downloaded file
3. Read the file .aqua-checksums.json on the same directory as aqua configuration file. If .aqua-checksums.json isn't found, aqua treats the file is empty
4. Get the expected checksum of downloaded file from .aqua-checksums.json
5. If the actual checksum is different from the expected checksum, make the installation failure. If the checksum isn't found in .aqua-checksums.json, the actual checksum is added to .aqua-checksums.json
6. Install the file

⚠️ Note

If the file is falsified before the correct checksum is added to .aqua-checksums.json, it is difficult to detect the falsification.

💡 Ignore the specific checksum verification

If you would like to ignore the specific checksum verification, you can do it by removing the checksum from .aqua-checksums.json.

.aqua-checksums.json

Pairs of package id and checksum are recorded.

e.g.

{
  "github_archive/github.com/tfutils/tfenv/v2.2.3": "0b42330aeed675ad3e5dd6063dbd1daabecb36180e515cca1c6e105dd7a1fa49"
}

This file is created and updated by aqua automatically, so you don't have to update this file manually.

Question: Should .aqua-checksums.json be managed with Git?

Coming soon

Algorithm to calculate checksums

sha256 is used for goreleaser by default, so aqua would also support sha256 by default.
In future, aqua may support other algorithms too.

Idea: command to update .aqua-checksums.json like terraform providers lock

Coming soon

Future work

• Verify Registry
• Verify aqua-proxy
• Support checksum algorithm other than sha256
• Get checksums from GitHub Releases's file

Implementation

• feat: support verifying downloaded assets with checksums #794

Reference

• https://news.ycombinator.com/item?id=29437903
• https://pkg.go.dev/crypto/sha256

[suzuki-shunsuke]

• https://zenn.dev/skanehira/articles/2021-04-18-go-hash

[suzuki-shunsuke]

How to configure

- type: github_release
  repo_owner: suzuki-shunsuke
  repo_name: github-comment
  asset: 'github-comment_{{trimV .Version}}_{{.OS}}_amd64.tar.gz'
  description: CLI to create a GitHub comment
  checksum:
    filename: 'github-comment_{{trimV .Version}}_checksums.txt'
    format: goreleaser
    algorithm: sha256
    # how to extract the checksum from file

[suzuki-shunsuke]

goreleaser

• https://goreleaser.com/customization/checksum/

  # Algorithm to be used.
  # Accepted options are sha256, sha512, sha1, crc32, md5, sha224 and sha384.
  # Default is sha256.
  algorithm: sha256

• https://github.com/suzuki-shunsuke/github-comment/releases/tag/v4.0.1
• https://github.com/suzuki-shunsuke/github-comment/releases/download/v4.0.1/github-comment_4.0.1_checksums.txt

9568289b4cabb368771b2cd92575b9474ced9865b092f13cecf992cfcf908bae  github-comment_4.0.1_linux_amd64.tar.gz
b5c06ff10364f136ccfa65e8bd0a4154bef94d79910fab41413b756c51520224  github-comment_4.0.1_windows_amd64.tar.gz
fb3b75af28078dbeef9c9dde95955fa7877623c8f77d3ebaaba4990764fb6c8e  github-comment_4.0.1_darwin_amd64.tar.gz

[suzuki-shunsuke]

aqua.checksum.json

{
  "packages": [
    {
      "name": "suzuki-shunsuke/github-comment@v4.0.0",
      "assets": [
        {
          "id": "github-comment_4.0.1_linux_amd64.tar.gz",
          "checksum": "9568289b4cabb368771b2cd92575b9474ced9865b092f13cecf992cfcf908bae"
        }
      ]
    }
  ]
}

Create or update file when aqua i is run.
Get checksum when file is donwloaded.

[suzuki-shunsuke]

I gave up this idea once, but I reconsider it.

[suzuki-shunsuke]

I've implemented basic feature by #794 .
It works as expected, but it is difficult to merge this for now because it harms the user experience of aqua.
We have to maintain .aqua-checksums.json.
When we update tools by Renovate, we have to update .aqua-checksums.json too.
The checksum would be different per OS and CPU architecture.
Some users would be confused by unexpected changes of .aqua-checksums.json.

We have to solve the problem.

[suzuki-shunsuke]

I'm working on this.

[suzuki-shunsuke]

aqua.yaml

checksum:
  enabled: true

    checksum:
      type: github_release
      algorithm: sha256
      path: tfcmt_{{trimV .Version}}_checksums.txt
      file_format: regexp
      pattern:
        checksum: ^(.{64})
        file: ^.{64}  (.*)$

tfcmt_3.2.5_checksums.txt

2397316c12b9e8be8756fde576607fe6d0eb7ddc2bc6ed78b55b787167847ecd  tfcmt_darwin_arm64.tar.gz
4e7951939337b45ff150d00b8cef8eba1c011232290d6902b0895740a20dfdb0  tfcmt_linux_arm64.tar.gz
652fced48841530601f7baef6150547820392c0498929e7a7d9a90c298d9abce  tfcmt_windows_arm64.tar.gz
90a36e8d0c373ff22755aa8231c3b37deb3e3f03d3f048017b6c51296114a4cf  tfcmt_windows_amd64.tar.gz
93899f8cce451c007740493f32187625418bebbe7ec9fa975cf865fa65bd5f06  tfcmt_darwin_amd64.tar.gz
e23730b18d1ae83242f0d0b48ceeb2db71f55b5ef44426cc2830f98f8b571a3c  tfcmt_linux_amd64.tar.gz