Build soci-snapshotter package

[larvacea]

Description of changes:
This builds the soci-snapshotter containerd plugin. The plugin has two components:

• soci-snapshotter-grpc is a daemon (systemd service).
• soci is a CLI tool for working with SOCI indices.

In addition, the rpm packages (and should install) soci-snapshotter.service, the unit file for systemd, so the daemon starts during boot (after network, and before containerd).

The aws-dev variant (and no other variant) includes the package.

Testing done:
[Edit] Manual testing on an aws-dev ec2 instance, following the soci-snapshotter installation and getting-started guides. The soci-snapshotter documentation uses nerdctl, so these packages were tested using nerdctl. That required a few extra steps:

• Create an overlay file system on the /root directory because nerdctl wants to store registry credentials in $HOME/.docker/config.json.
• Install wget and tar in the admin container, download, then copy nerdctl over to the host.
• Add the soci-snapshotter plugin to containerd's configuration (in the AMI)

Following the steps outlined in the soci-snapshotter repo, I can verify containerd will happily pull and run an image through soci. I see exactly the transactions I expect with ECR and the plugin is able to find and download the soci index, ztoc, and manifest files given the package URI. Both pull and run benefit from lazy loading on the test image they suggested.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[yeazelm]

Per your testing comments:

Absolutely none. This definitely builds the RPM, and the rpm tools say it is all perfectly fine. The aws-dev variant builds. If one wanted to inform containerd of the plugin's existence, one would need to add a few lines to the containerd config.toml file.

Can we try a bit more validation than this? Since it is a dev image, I don't think we need to fully integrate the API for the configuration settings, but at least you could attempt to get containerd to use it via the shell manually? Just because it builds into an rpm doesn't mean the resulting binaries are usable. I'd like to at least see a basic smoke test to ensure the binaries could work once the rest of the configuration is provided.

[larvacea]

Testing confirms that soci-snapshotter (running on bottlerocket, as a systemd service) works with or without fuse. Without fuse, soci-snapshotter will write error messages to the journal.

[bcressey]

Pretty sure the snapshotter binaries will need to be built for FIPS and non-FIPS, so I've suggested the changes required for that.

[larvacea]

I have incorporated all of Ben's suggestions except changing the source URL. Changing the source URL broke the build, and I have not yet debugged that problem.

[ducminhle]

Hi @larvacea,
Is this PR ready for EKS?
When I install soci-snapshotter on EKS (with pre-boostrap user-data), I need to install amazon-ecr-credential-helper and config $HOME/.docker/config.json to pull images from ECR.

{
  "credsStore": "ecr-login"
}

[arnaldo2792]

Hey @ducminhle , this is not ready to be used by EKS or ECS yet, due to the dependencies that you described. We need to work closely with the EKS and SOCI maintainers to figure out if there is a different solution to support credentials in SOCI.

[ducminhle]

Hey @ducminhle , this is not ready to be used by EKS or ECS yet, due to the dependencies that you described. We need to work closely with the EKS and SOCI maintainers to figure out if there is a different solution to support credentials in SOCI.

Thank you for your answer