Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

composefs: enable it at runtime #1769

Merged
merged 5 commits into from Dec 13, 2023
Merged

composefs: enable it at runtime #1769

merged 5 commits into from Dec 13, 2023

Conversation

giuseppe
Copy link
Member

@giuseppe giuseppe commented Dec 6, 2023

drop the build tag for building composefs and convert it to an option in storage.conf

@openshift-ci openshift-ci bot added the approved label Dec 6, 2023
@giuseppe
Copy link
Member Author

giuseppe commented Dec 6, 2023

@cgwalters PTAL

@giuseppe giuseppe force-pushed the always-build-composefs branch 5 times, most recently from afafaa4 to 3869129 Compare December 6, 2023 11:22
@cgwalters
Copy link
Contributor

So I was trying this out like this:

$ podman --log-level debug --storage-opt use_composefs=true --root /var/tmp/containers --runroot /run/tmp/containers run --rm -ti quay.io/fedora/fedora:39 echo hello

I see

DEBU[0000] overlay: use_composefs=true 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that data-only layers for overlay are supported 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is not being used

Yet AFAICS generateComposeFsBlob() is never being invoked because ApplyDiffFromStagingDirectory is never being invoked? But I don't understand the conditions under which that is invoked.

@giuseppe
Copy link
Member Author

giuseppe commented Dec 6, 2023

Yet AFAICS generateComposeFsBlob() is never being invoked because ApplyDiffFromStagingDirectory is never being invoked? But I don't understand the conditions under which that is invoked.

it works only with the zstd:chunked code path now, you need to add this snippet to storage.conf:

[storage.options]

pull_options = {convert_images = "true", enable_partial_images = "true", use_hard_links = "false", ostree_repos=""}

@cgwalters cgwalters changed the title composefs: enable it a runtime composefs: enable it at runtime Dec 6, 2023
@cgwalters
Copy link
Contributor

cgwalters commented Dec 6, 2023 •
edited

Ah, OK only operating on zstd:chunked and requiring an opt-in and previously a build tag definitely makes it hard to use 😄
Removing one of those things helps...actually, the build tag thing seems like it'd make doing CI tests against this much easier, right?

Anyways I think I've done what you suggested, but I am still not managing to trigger the composefs path. I do see

DEBU[0000] Found image "83a6861c2dbd64301df6dde82f9d99b3659b50478fa9cb7296e21552212a3757" as "83a6861c2dbd64301df6dde82f9d99b3659b50478fa9cb7296e21552212a3757" in local containers storage ([overlay@/var/tmp/containers/storage+/run/tmp/containers:overlay.mountopt=nodev,metacopy=on,overlay.use_composefs=true]@83a6861c2dbd64301df6dde82f9d99b3659b50478fa9cb7296e21552212a3757)

which is different, but apparently ApplyDiffFromStagingDirectory still isn't being run.

@giuseppe
Copy link
Member Author

you already had the image in the storage? Was it re-pulled once you enabled the storage options?

@giuseppe
tests: update golangci-lint
44ffef4 
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@giuseppe
composefs: trim / from paths
af91fe3 
it solves the following error when pulling the alpine image:

mkcomposefs: Parent directory missing for /bin/

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@giuseppe
config: new option use_composefs
ca642ac 
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@giuseppe
overlay: always build composefs
6c3d795 
and honor the use_composefs configuration from the storage.conf file.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@giuseppe
overlay: inhibit filegetter with composefs
17bc7c5 
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@giuseppe
Copy link
Member Author

@rhatdan PTAL

@rhatdan
Copy link
Member

rhatdan commented Dec 13, 2023

LGTM
@cgwalters @eriksjolund @nalind @vrothberg @saschagrunert PTAL

saschagrunert
saschagrunert approved these changes Dec 13, 2023
View reviewed changes
Copy link
Member

@saschagrunert saschagrunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

drivers/overlay/overlay.go
@@ -387,6 +389,22 @@ func Init(home string, options graphdriver.Options) (graphdriver.Driver, error)
}
}

if opts.useComposefs {
if unshare.IsRootless() {
return nil, fmt.Errorf("composefs is not supported in user namespaces")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non blocking nit:

Suggested change
return nil, fmt.Errorf("composefs is not supported in user namespaces")
return nil, errors.New("composefs is not supported in user namespaces")

drivers/overlay/overlay.go
@@ -122,6 +123,7 @@ type Driver struct {
supportsDType bool
supportsVolatile *bool
usingMetacopy bool
usingComposefs bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking nit:

Suggested change
usingComposefs bool
useComposefs bool

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 13, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: giuseppe, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [giuseppe,saschagrunert]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rhatdan
Copy link
Member

rhatdan commented Dec 13, 2023

@giuseppe could you fix @saschagrunert Nit's

@rhatdan
Copy link
Member

rhatdan commented Dec 13, 2023

I am going to merge to fix CI/CD system on other PRs. @giuseppe if you care to fix the nits, please open a new PR.

@rhatdan rhatdan merged commit 6d05a39 into containers:main Dec 13, 2023
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eriksjolund eriksjolund eriksjolund left review comments

@saschagrunert saschagrunert saschagrunert approved these changes

Assignees
No one assigned
Labels
approved
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@giuseppe @cgwalters @rhatdan @eriksjolund @saschagrunert