Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oci-sif: add --keep-layers to preserve layers in OCI-SIF creation #2273

Closed
wants to merge 1 commit into from

Conversation

dtrudg
Copy link
Member

@dtrudg dtrudg commented Oct 16, 2023

When an OCI-SIF is created explicitly with pull, or implicitly with run/shell/exec/instance start, permit the image layers to be kept intact.

The new --keep-layers flag for these commands skips squashing images to a single layer when an OCI-SIF is created from an OCI source.

This is the initial step for multi-layered OCI-SIF creation. It is not possible to execute the images. Testing will be added as we proceed to refactor a lot of image handling code to allow execution of these multi-layer images.

$ singularity pull --oci --keep-layers docker://golang
FATAL:   Image file already exists: "golang_latest.oci.sif" - will not overwrite

$ singularity sif list golang_latest.oci.sif
------------------------------------------------------------------------------
ID   |GROUP   |LINK    |SIF POSITION (start-end)  |TYPE
------------------------------------------------------------------------------
1    |1       |NONE    |32176-47541680            |OCI.Blob
2    |1       |NONE    |47541680-66674096         |OCI.Blob
3    |1       |NONE    |66674096-124489136        |OCI.Blob
4    |1       |NONE    |124489136-214961584       |OCI.Blob
5    |1       |NONE    |214961584-281157040       |OCI.Blob
6    |1       |NONE    |281157040-281161136       |OCI.Blob
7    |1       |NONE    |281161136-281162124       |OCI.Blob
8    |1       |NONE    |281162124-281163339       |OCI.Blob
9    |1       |NONE    |281163339-281163580       |OCI.RootIndex

Fixes #2266

When an OCI-SIF is created explicitly with `pull`, or implicitly with
`run/shell/exec/instance start`, permit the image layers to be kept
intact.

The new `--keep-layers` flag for these commands skips squashing images
to a single layer when an OCI-SIF is created from an OCI source.

This is the initial step for multi-layered OCI-SIF creation. It is not
possible to execute the images. Testing will be added as we proceed to
refactor a lot of image handling code to allow execution of these
multi-layer images.

```
$ singularity pull --oci --keep-layers docker://golang
FATAL:   Image file already exists: "golang_latest.oci.sif" - will not overwrite

$ singularity sif list golang_latest.oci.sif
------------------------------------------------------------------------------
ID   |GROUP   |LINK    |SIF POSITION (start-end)  |TYPE
------------------------------------------------------------------------------
1    |1       |NONE    |32176-47541680            |OCI.Blob
2    |1       |NONE    |47541680-66674096         |OCI.Blob
3    |1       |NONE    |66674096-124489136        |OCI.Blob
4    |1       |NONE    |124489136-214961584       |OCI.Blob
5    |1       |NONE    |214961584-281157040       |OCI.Blob
6    |1       |NONE    |281157040-281161136       |OCI.Blob
7    |1       |NONE    |281161136-281162124       |OCI.Blob
8    |1       |NONE    |281162124-281163339       |OCI.Blob
9    |1       |NONE    |281163339-281163580       |OCI.RootIndex
```

Fixes sylabs#2266
@dtrudg dtrudg self-assigned this Oct 16, 2023
@dtrudg dtrudg marked this pull request as ready for review October 16, 2023 12:58
@dtrudg dtrudg marked this pull request as draft October 17, 2023 12:13
@dtrudg
Copy link
Member Author

dtrudg commented Oct 20, 2023

Closing - subsumed into #2274

@dtrudg dtrudg closed this Oct 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support pull from OCI registry to a multi-layer OCI-SIF
1 participant