Add --chown option to add/copy #336

bertinatto · 2017-11-27T14:59:02Z

Hi there,

I came across a permission error while I was automating the migration of a Redis setup into a container. Basically, buildah copy was copying my file into the container and leaving it owned by root.

While I know that an extra buildah run containerID chown redis:redis /etc/redis.conf would solve that issue, I would love to have a --chown option in buildah add/copy. I know that docker added something similar recently, but I haven't looked into their implementation.

This PR is simply a PoC to to get a feedback from you. It still needs some improvements, so if you think I can go ahead, I'll look into adding in some tests, docs and refactor the code.

This is an example of how it works:

# Create container
$ sudo ./buildah from registry.access.redhat.com/rhscl/redis-32-rhel7
redis-32-rhel7-working-container-15

# Copy some data into the container, setting the owner
$ sudo ./buildah copy --chown redis:redis redis-32-rhel7-working-container-15 contrib /data/contrib
$ sudo ./buildah copy --chown redis:redis redis-32-rhel7-working-container-15 add.go /data/
$ sudo ./buildah copy --chown redis:redis redis-32-rhel7-working-container-15 add.go /data/
$ sudo ./buildah copy --chown redis:redis redis-32-rhel7-working-container-15 https://github.com/projectatomic/buildah/blob/master/README.md /data/

# Owner is properly set
$ sudo ./buildah run redis-32-rhel7-working-container-15 sh
sh-4.2$ ls /data -l
total 64
-rw-------. 1 redis redis 45981 Nov 27 14:41 README.md
-rw-rw-r--. 1 redis redis  8779 Nov 27 14:35 add.go
drwxr-xr-x. 4 redis redis  4096 Nov 27 14:40 contrib
sh-4.2$ ls /data/contrib/ -l
completions/ rpm/
sh-4.2$ ls /data/contrib/rpm/buildah.spec -l
-rw-rw-r--. 1 redis redis 5347 Nov 23 13:49 /data/contrib/rpm/buildah.spec
sh-4.2$

Thanks for looking into this!

rh-atomic-bot · 2017-11-27T15:01:00Z

Can one of the admins verify this patch?
I understand the following commands:

bot, add author to whitelist
bot, test pull request
bot, test pull request once

TomSweeneyRedHat · 2017-11-27T15:19:45Z

@bertinatto Thanks for the PR/PoC. The change sounds good to me, but I'll let @nalind and @rhatdan give the definitive answer. If we do move forward, we'll want to add some tests and man page changes too.

The test failure is not related to this change. @rhatdan the error is these two versions aren't matching:

# buildah version: 0.8
# buildah rpm version: 0.7

Does buildah.spec need to have it's version bumped? It's set at 0.7 atm.

rhatdan · 2017-11-27T15:38:30Z

bot, add author to whitelist

rhatdan · 2017-11-27T15:41:09Z

Since this made it into Docker, I agree that it should be implemented.

rhatdan · 2017-11-27T15:41:16Z

moby/moby#34263

rhatdan · 2017-11-27T15:48:59Z

add.go

+
+				// Set permissions. For now both uid and gid need to be set and they have to be numbers.
+				if options.Chown[0] != "" && options.Chown[1] != "" {
+					// TODO: figure out uid/id if user/group names was passed in instead of the real uid/gid.


I believe there is code for this already written to handle passing of of USER string.

rhatdan · 2017-11-27T15:51:34Z

add.go

+//AddAndCopyOptions holds options for add and copy commands.
+type AddAndCopyOptions struct {
+	// These are strings because I want to accept both ids and names.
+	Chown [2]string


Since you have a struct, why not make it
USER string,
Group String,

rhatdan · 2017-11-27T15:53:07Z

add.go

+					}
+
+					if err := os.Chown(d, uid, gid); err != nil {
+						return errors.Wrapf(err, "error setting permissions")


error changing ownership %s:%s", options.Chown[0], options.Chown[1]

Although I would prefer options.User, options,Group

rhatdan · 2017-11-27T15:53:47Z

cmd/buildah/addcopy.go

+	addAndCopyFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "chown",
+			Usage: "Set the user and group ownership of the file",


of the destination content.

rhatdan · 2017-11-27T15:56:48Z

cmd/buildah/addcopy.go

@@ -52,7 +67,16 @@ func addAndCopyCmd(c *cli.Context, extractLocalArchives bool) error {
 		return errors.Wrapf(err, "error reading build container %q", name)
 	}

-	err = builder.Add(dest, extractLocalArchives, args...)
+	options := buildah.AddAndCopyOptions{}
+	if chown := c.String("chown"); chown != "" {


Why does this have to be two, I would think you should just do a
if c.String("chown") != "" {
Then split it. I don't think we need to care of group is not set.

--chown 1234
Should be ok, and then just leave the other component alone.
}

nalind · 2017-11-28T21:58:07Z

add.go

@@ -69,6 +78,12 @@ func (b *Builder) Add(destination string, extract bool, source ...string) error
 		}
 	}()
 	dest := mountPoint
+
+	uid, gid, err := findUserGroupIDs(mountPoint, options.Chown)


I think it'd be better to use getUser() from user.go here.

Wow, too bad I hadn't see getUser() before! Thanks for pointing this out.

bertinatto · 2017-12-01T19:21:02Z

Thank you for the initial look!

I've made a few adjustments, added a few tests and updated the man pages. Would you mind taking another look?

The second commit is just to make the tests pass. I tried to make my code simpler to avoid this change, but I' afraid there isn't much I can do. Any suggestions?

TomSweeneyRedHat · 2017-12-01T20:23:47Z

add.go

@@ -68,6 +74,18 @@ func (b *Builder) Add(destination string, extract bool, source ...string) error
 			logrus.Errorf("error unmounting container: %v", err2)
 		}
 	}()
+	// Set ownership of the destination.  If this information is


It's a fairly long time before the user variable is used from this point and a number of other errors that could happen before then. I'm just wondering if it would make sense to move this new block of code down to be run just before the for statement on line 121.

That's right, I missed that!

TomSweeneyRedHat · 2017-12-01T20:26:37Z

tests/validate/gometalinter.sh

@@ -16,6 +16,6 @@ exec gometalinter.v1 \
 	--disable=gotype \
 	--disable=gas \
 	--disable=aligncheck \
-	--cyclo-over=40 \
+	--cyclo-over=45 \


Not sure what this change helps with?

Commit 9227692 introduced a few ifs in the Add() function, increasing it its "cyclomatic complexity". As a result, make validate was failing with:

add.go:67::warning: cyclomatic complexity 41 of function (*Builder).Add() is high (> 40) (gocyclo)

I increased the limit in gometalinter.sh only to make the tests pass, however, I know this isn't a proper solution. Any suggestions on how to address this?

rhatdan · 2017-12-03T12:28:23Z

add.go

@@ -100,6 +106,18 @@ func (b *Builder) Add(destination string, extract bool, source ...string) error
 	if len(source) > 1 && (destfi == nil || !destfi.IsDir()) {
 		return errors.Errorf("destination %q is not a directory", dest)
 	}
+	// Find out which user/group should the destination belong to.


Create a separate function

func (b *Builder) user(options AddAndCopyOptions) (string, error) { // If this information is passed in through --chown, use it; // otherwise use the user/group as whom the container should run. if options.Chown != "" { return getUser(mountPoint, options.Chown) } return getUser(mountPoint, b.User()) }

That's right! Thanks!

rhatdan · 2017-12-05T12:28:04Z

LGTM
@nalind WDYT?

TomSweeneyRedHat · 2017-12-05T14:06:47Z

add.go

@@ -100,6 +106,11 @@ func (b *Builder) Add(destination string, extract bool, source ...string) error
 	if len(source) > 1 && (destfi == nil || !destfi.IsDir()) {
 		return errors.Errorf("destination %q is not a directory", dest)
 	}
+	// Find out which user (and group) should the destination belong to.


nit of a nit. "should the destination" to "the destination should". But don't worry about it unless there are other needed changes.

TomSweeneyRedHat

LGTM, minor nit.

nalind · 2017-12-05T14:43:28Z

add.go

+	}
+	if fi.IsDir() {
+		err2 := filepath.Walk(path, func(p string, info os.FileInfo, we error) error {
+			if err3 := os.Chown(p, int(user.UID), int(user.GID)); err3 != nil {


os.Chown() follows symlinks, and since we're not doing any of this in a chrooted subprocess, we could open ourselves up to something like #66 again. I think using os.Lchown() is a first step here.

Hm, interesting perspective, thank you!

Edit: by the way, fixed.

bertinatto · 2017-12-05T15:27:24Z

Failing test seems to be unrelated to this patch:

# error pulling image "registry.fedoraproject.org/fedora:27": Error initializing image from source docker://registry.fedoraproject.org/fedora:27: pinging docker registry returned: error pinging repository, response code 503

TomSweeneyRedHat · 2017-12-05T15:39:48Z

@bertinatto I concur that it's unrelated. The fedora site seems to be hiccuping like crazy today. I've seen this error in other PR's this morning and I am having difficulty pulling kits from there outside of GitHub.

Signed-off-by: Fabio Bertinatto <fbertina@redhat.com>

bertinatto · 2017-12-07T10:32:12Z

Cool, tests are passing now!

rhatdan · 2017-12-07T13:44:54Z

Ok I will merge, the question is do we need more protection with the chown call then just lchown.

rhatdan · 2017-12-07T13:45:09Z

@bertinatto thanks for this.
@rh-atomic-bot r+

rh-atomic-bot · 2017-12-07T13:45:10Z

📌 Commit b81b52d has been approved by rhatdan

rh-atomic-bot · 2017-12-07T13:45:14Z

⚡ Test exempted: pull fully rebased and already tested.

rhatdan reviewed Nov 27, 2017

View reviewed changes

nalind reviewed Nov 28, 2017

View reviewed changes

bertinatto force-pushed the chown branch 11 times, most recently from f1724c4 to dcb6c85 Compare December 1, 2017 18:43

TomSweeneyRedHat reviewed Dec 1, 2017

View reviewed changes

bertinatto force-pushed the chown branch from dcb6c85 to ec8f883 Compare December 2, 2017 18:40

rhatdan reviewed Dec 3, 2017

View reviewed changes

bertinatto force-pushed the chown branch from ec8f883 to c0ce49d Compare December 3, 2017 19:42

bertinatto changed the title ~~[WIP] Add --chown option to add/copy~~ Add --chown option to add/copy Dec 3, 2017

TomSweeneyRedHat reviewed Dec 5, 2017

View reviewed changes

nalind reviewed Dec 5, 2017

View reviewed changes

bertinatto force-pushed the chown branch from c0ce49d to 695cf50 Compare December 5, 2017 15:09

Add --chown option to add/copy commands

b81b52d

Signed-off-by: Fabio Bertinatto <fbertina@redhat.com>

bertinatto force-pushed the chown branch from 695cf50 to b81b52d Compare December 7, 2017 10:12

rh-atomic-bot closed this in 1fc5a49 Dec 7, 2017

bertinatto mentioned this pull request Jan 13, 2018

Don't overwrite directory permissions on --chown #389

Closed

github-actions bot added the locked - please file new issue/PR label Oct 28, 2023

github-actions bot locked as resolved and limited conversation to collaborators Oct 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add --chown option to add/copy #336

Add --chown option to add/copy #336

bertinatto commented Nov 27, 2017

rh-atomic-bot commented Nov 27, 2017

TomSweeneyRedHat commented Nov 27, 2017 •

edited

Loading

rhatdan commented Nov 27, 2017

rhatdan commented Nov 27, 2017

rhatdan commented Nov 27, 2017

rhatdan Nov 27, 2017

rhatdan Nov 27, 2017

rhatdan Nov 27, 2017

rhatdan Nov 27, 2017

rhatdan Nov 27, 2017

nalind Nov 28, 2017 •

edited

Loading

bertinatto Nov 30, 2017

bertinatto commented Dec 1, 2017

TomSweeneyRedHat Dec 1, 2017

bertinatto Dec 2, 2017

TomSweeneyRedHat Dec 1, 2017

bertinatto Dec 2, 2017

rhatdan Dec 3, 2017

bertinatto Dec 5, 2017

rhatdan commented Dec 5, 2017

TomSweeneyRedHat Dec 5, 2017

TomSweeneyRedHat left a comment

nalind Dec 5, 2017

bertinatto Dec 5, 2017 •

edited

Loading

bertinatto commented Dec 5, 2017

TomSweeneyRedHat commented Dec 5, 2017

bertinatto commented Dec 7, 2017

rhatdan commented Dec 7, 2017

rhatdan commented Dec 7, 2017

rh-atomic-bot commented Dec 7, 2017

rh-atomic-bot commented Dec 7, 2017

Add --chown option to add/copy #336

Add --chown option to add/copy #336

Conversation

bertinatto commented Nov 27, 2017

rh-atomic-bot commented Nov 27, 2017

TomSweeneyRedHat commented Nov 27, 2017 • edited Loading

rhatdan commented Nov 27, 2017

rhatdan commented Nov 27, 2017

rhatdan commented Nov 27, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nalind Nov 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bertinatto commented Dec 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhatdan commented Dec 5, 2017

Choose a reason for hiding this comment

TomSweeneyRedHat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bertinatto Dec 5, 2017 • edited Loading

Choose a reason for hiding this comment

bertinatto commented Dec 5, 2017

TomSweeneyRedHat commented Dec 5, 2017

bertinatto commented Dec 7, 2017

rhatdan commented Dec 7, 2017

rhatdan commented Dec 7, 2017

rh-atomic-bot commented Dec 7, 2017

rh-atomic-bot commented Dec 7, 2017

TomSweeneyRedHat commented Nov 27, 2017 •

edited

Loading

nalind Nov 28, 2017 •

edited

Loading

bertinatto Dec 5, 2017 •

edited

Loading