Validation failures files created when pushing images with zstd compressed layers

[diegmonti]

Description

When pushing Docker images to Sonatype Nexus with layers compressed using the Zstandard (zstd) compression algorithm, validation failures files are being created in the /nexus-data/tmp directory.

Steps to reproduce

1. Create a Dockerfile with multiple layers:

FROM public.ecr.aws/docker/library/alpine:latest

RUN echo hello > /tmp/test
RUN echo world >> /tmp/test

2. Set up a Docker buildx builder:

$ docker buildx create --use --name nexus-zstd-demo

3. Build the Docker image with zstd compression and push it to Nexus:

$ docker buildx build --output type=image,name=registry.example.com/nexus-zstd-demo,push=true,compression-level=3,force-compression=true,compression=zstd .

4. Check the contents of the /nexus-data/tmp directory.

Expected behavior

No validation failures files should be created in the /nexus-data/tmp directory when pushing images with zstd-compressed layers to Nexus.

Observed behavior

After pushing the Docker image with zstd-compressed layers to Nexus, validation failures files are generated in the /nexus-data/tmp directory. These files appear to correspond to the layers of the pushed image, indicating that the validation process encountered issues with these layers.

Environment

Sonatype Nexus version 3.67.1 deployed as Docker image on Kubernetes.

Attachments

Output of the ls -l /nexus-data/tmp command showing the validation failures files:

-rw------- 1 nexus nexus  123 Apr 24 13:37 docker-content-validation-failures1849768073000605819
-rw------- 1 nexus nexus  143 Apr 24 13:37 docker-content-validation-failures7179895501149398454

[nblair]

Hi @diegmonti - thanks for opening an issue. I'm unfamiliar with 'zstd-compressed' layers. What tools would be used to generate layers in this way? What benefits does this capability offer, or what requirements does it satisfy?

I've labeled this as an enhancement request, as I don't expect we have built in support for it.

[diegmonti]

Hi @nblair!

Zstandard is a fast compression algorithm designed to provide high compression ratios with low latency. In the context of Docker images, zstd compression can be applied to individual layers of the image during the build process. Starting from version 23, Docker supports zstd compression with BuildKit.

The benefits of using zstd compression for Docker image layers include: improved compression efficiency, reduced image size and faster decompression times.

It seems that Nexus is capable of handling layers compressed with zstd without issues in terms of pushing and pulling them. The only concern lies in the creation of validation failure files.