cmd/snap-update-ns: add actual implementation #3225

zyga · 2017-04-24T11:50:19Z

This patch adds a non-dummy implementation of snap-update-ns. There are
still three pieces missing. There's no locking so concurrently running
snap-confine is not synchronized. The function that determines if a
mount change is needed is dummy and always returns true. The mount
changes are not really performed yet as the Perform function is just a
stub. The stubs will be addressed with separate PRs.

All that the tool now does is to print what should be done instead of
actually doing it.

Signed-off-by: Zygmunt Krynicki zygmunt.krynicki@canonical.com

zyga · 2017-04-24T21:48:48Z

I found a small bug that required changes to ChangesNeeded. I'll update the branch shortly.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

This assists in computing the effective current profile as all the kept and mounted things are in in. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

This patch adds a non-dummy implementation of snap-update-ns. There are still two pieces missing. The function that determines if a mount change is needed is dummy and always returns true. The mount changes are not really performed yet as the Perform function is just a stub. The stubs will be addressed with separate PRs. All that the tool now does is to print, to stdout, what should be done instead of actually doing it. Stderr is a bit more noisy, but essentially explains the same thing with more detail. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

stolowski

Just made 1st quick pass over these changes. Looks good, some nitpicks, see individual comments, will do 2nd pass later.

It seems that we will be keeping .lock file around forever, which is fine... Just curious if there are strong reason s to do that, instead of creating them with O_EXCL and removing when done?

stolowski · 2017-04-25T09:35:58Z

interfaces/mount/lock.go

+
+// lockFileName returns the name of the lock file for the given snap.
+func lockFileName(snapName string) string {
+	return filepath.Join(dirs.SnapRunLockDir, fmt.Sprintf("%s.lock", snapName))


I wonder if we will ever want more lock files for other non-conflicting operations, in which case it would make sense to give this lock a more specific name, e.g. snap.mount-lock?

So far all locking is either global (all namespaces) or scoped to a specific snap. The lock file protects the $SNAP_NAME.mnt file from concurrent modification.

stolowski · 2017-04-25T09:39:34Z

cmd/snap-update-ns/main.go

+	changesNeeded := mount.NeededChanges(current, desired)
+	fmt.Fprintf(os.Stderr, "CHANGES NEEDED:\n")
+	for _, change := range changesNeeded {
+		fmt.Fprintf(os.Stderr, " - %s\n", change)


How about a small lambda to avoid the repetitions of fmt.Fprintf(os.Stderr, " - %s\n".... above and below? The lambda could possibly replace the entire loop, but I'm not sure of that.

Oh, I'll just drop those. I don't think we need them.

Dropped now.

stolowski · 2017-04-25T09:42:13Z

tests/main/snap-update-ns/task.yaml

+    # current mount namespace.
+    /usr/lib/snapd/snap-discard-ns $PLUG_SNAP
+    echo "Check that snap-update-ns fails after discarding the mount namespace"
+    /usr/lib/snapd/snap-update-ns $PLUG_SNAP 2>snap-update-ns.log | MATCH "cannot update snap namespace: cannot switch mount namespace: invalid argument"


Very nice, thanks for all these tests!

I have way more coming. This works but I have also the full-blown version that does everything automatically and I'll be just adding more tests now.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

zyga · 2017-04-25T11:43:48Z

We cannot remove the lock files as this would make them useless. If we open them with exclusive flag then only one process can succeed and ... what then? What does the 2nd guy do? Try again? The trick is that nobody removes them (maybe snapd could when the snap is purged entirely) so that anyone can open them and then the real race is around the only primitive that is sensible, flock itself.

stolowski · 2017-04-25T12:12:26Z

cmd/snap-update-ns/main.go

 	// There is some C code that runs before main() is started.
 	// That code always runs and sets an error condition if it fails.
 	// Here we just check for the error.
 	if err := BootstrapError(); err != nil {
+		// If there is no mount namespace to transition to let's just quit
+		// instantly without any errors as there is nothing to do anymore.


Please bear with me and excuse me my ignorance... Can you explain why not having a mount ns to transition to is ok here and can be silently ignored? Perhaps extending this comment to explain what is the typical scenario for this to happen would be good for anyone not familiar with namespaces :}

The goal of the tool is to update a mount namespace. If no mount namespace exists there is nothing to do

This essentially allows snapd to just use this tool without having to coordinate

niemeyer

Thanks, glad to see the feature almost there.

niemeyer · 2017-04-25T14:51:35Z

cmd/snap-update-ns/bootstrap.go

 	"fmt"
 	"syscall"
 	"unsafe"
 )

+var (
+	ErrNoNS = errors.New("no namespace")
+)


This can be a single line, and it'd be nice to have a still terse message but slightly more clear one so that if it ever leaks we know where to look at:

var ErrNoNS = errors.New("cannot find namespace to update")

+1, will change

niemeyer · 2017-04-25T14:57:11Z

cmd/snap-update-ns/main.go

+	// of snap-confine are synchronized and will see consistent state.
+	lock, err := mount.OpenLock(snapName)
+	if err != nil {
+		return fmt.Errorf("cannot open mount namespace lock file: %s", err)


Thanks for the descriptive errors here and below!

Oh, can we please add the snap name to all of these errors? This will definitely be helpful when debugging.

"cannot open mount namespace lock file for snap %q: %s"

etc.

niemeyer · 2017-04-25T14:57:44Z

cmd/snap-update-ns/main.go

+	if err := lock.Lock(); err != nil {
+		return fmt.Errorf("cannot lock mount namespace: %s", err)
+	}
+	defer lock.Close()


This should be before the branch above.

Ah, good point.

niemeyer · 2017-04-25T15:03:37Z

cmd/snap-update-ns/main.go

+			changesMade = append(changesMade, change)
+			continue
+		}
+		// Read mount info each time as our operations may have unexpected


That seems awkward. Doing that when something errors is perhaps justifiable since we don't know whether it worked or not, but loading it every single time because we have no idea seems very suspect.

I think it is ok to err on the safe side. The alternative is to say the we know exactly how the kernel (including bugs) performs mount and unmount operations so that we can simulate them here. I'm not sure I like that assumption.

I'm still not comfortable with that. It's akin to rebooting the system because one has absolutely no clue of what is going on. Yes, it tends to work, but it demonstrates lack of understanding of the system, and problems that are being ignored.

If we need to reload this on every iteration, we very much need to know why we're doing that. What is changing between each of these iterations that could modify something that will affect follow up iterations? If the answer is we don't know, we need to think harder about what this tool is doing.

This part is now gone, along with Change.Needed

niemeyer · 2017-04-25T15:04:26Z

cmd/snap-update-ns/main.go

+		if err != nil {
+			return fmt.Errorf("cannot read mount-info table: %s", err)
+		}
+		if !change.Needed(mounted) {


Shouldn't this consider prefixes as well? I don't recall seeing that logic in Needed.

Can you expand on this? I think one thing we need to handle better here is when an operation fails we should abort all the changes to the sub-tree (e.g. don't try to mount something when earlier unmount in the same sub-tree failed). Is that what you mean?

What happens if mounted is a prefix of the modification described in change, and what should happen?

Aha, interesting! I think that the algorithm that computes the needed changes already handles prefix changes. Since I removed the Change.Needed code entirely I think this is okay now. We just do exactly what we computed and we always keep track of what we did.

niemeyer · 2017-04-25T15:04:42Z

cmd/snap-update-ns/main.go

+			changesMade = append(changesMade, change)
+			continue
+		}
+		fmt.Printf("%s\n", change)


In this version it is used for trivial testing. It gets removed when the Change.Perform branch is combined with a more extensive tests that measures actual mounts being changed, not just this being printed.

Oh, since Change.Perform branch has been merged I can iterate on this. Let me update the tests to do real stuff now.

That's still TODO?

No, it's done now.

It's still in the PR. We shouldn't be printing random output like this.

This is now gone :)

niemeyer · 2017-04-25T15:06:17Z

cmd/snap-update-ns/main.go

+
+	// Compute the new current profile so that it contains only changes that were made
+	// and save it back for next runs.
+	current = &mount.Profile{}


var current mount.Profile

But this is re-setting the existing variable, am I missing anything?

I renamed current to currentBefore and currentAfter so that there's no confusion about this. Also applied the suggestion you made.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

…e-ns/tool

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

stolowski

Looks good, just two comments regarding tests.

stolowski · 2017-04-28T10:20:52Z

cmd/snap-update-ns/main.go

+	// of snap-confine are synchronized and will see consistent state.
+	lock, err := mount.OpenLock(snapName)
+	if err != nil {
+		return fmt.Errorf("cannot open lock file for mount namespace of snap %q: %s", snapName, err)


It would be good to have a test for this error case, can you add one?

I'm working on a branch with unit tests for all of the code here.

stolowski · 2017-04-28T10:22:49Z

cmd/snap-update-ns/main.go

+			changesMade = append(changesMade, change)
+			continue
+		}
+		fmt.Printf("%s\n", change)


That's still TODO?

zyga · 2017-04-28T10:27:11Z

@stolowski it is not a todo, it is used by tests (the printf)

as for missing tests I think that testing the locking error is possible but as you see there are no unit tests at all here, just integration tests. I will be iterating on this (primarily on testing) but I'd love to see this land so that we can start testing it the hard way to discover the more interesting bugs.

stolowski

Ok, sure. Looking forward for the upcoming branches then. +1

…e-ns/tool

This patch adds a non-dummy implementation of snap-update-ns. There are still two pieces missing. The function that determines if a mount change is needed is dummy and always returns true. The mount changes are not really performed yet as the Perform function is just a stub. The stubs will be addressed with separate PRs. All that the tool now does is to print, to stdout, what should be done instead of actually doing it. Stderr is a bit more noisy, but essentially explains the same thing with more detail. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

The snap-update-ns tool used to print things so that initial tests could measure that something was going on. As the tool does everything now and runs automatically tests can be simplified to look for real side-effects instead. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

This patch removes the unimplemented `Change.Needed` method. The method was designed to inspect the mount namespace, as exposed by the kernel mountinfo interface, and look for signs that a change has already occured but was not recorded (e.g. it was constructed by version of snap-confine earlier than 2.25). In retrospective this feature is very complex and not really needed as we know exactly what was mounted so we don't need to guess (using the much more complex kernel interface). Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

chipaca

Mostly LGTM; just a couple of minor things

chipaca · 2017-05-15T12:33:17Z

tests/main/snap-update-ns/task.yaml

+    # Check that the shared content is not mounted.
+    snap run --shell $PLUG_SNAP.content-plug -c 'test ! -e $SNAP/import/shared-content'
+
+    # Run snap-update-ns to see that setns part worked and we got did nothing at all.


we got did nothing at all

this sentence has uses too many verbs

chipaca · 2017-05-15T12:36:46Z

cmd/snap-update-ns/bootstrap.go

-// Error returns error (if any) encountered in pre-main C code.
+var (
+	// ErrNoNS is a distinct error returned when a snap namespace does not exist.
+	ErrNoNS = errors.New("cannot update mount namespace that was not created yet")


I think we're trying to have all errors called FooError, not ErrFoo.

(I was wrong, as it's a variable and not a type)

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

Changes applied as requested. Gustavo is off for two days and I'd like to iterate. Chipaca approved

zyga force-pushed the feature/update-ns/tool branch from 5a18ba3 to 1544d5b Compare April 24, 2017 12:51

zyga added 4 commits April 25, 2017 00:17

dirs: add snap lock directory

b579fa8

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

interfaces/mount: add support for locking namespaces

91585b6

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

interfaces/mount: spell unmount correctly

ba19bbf

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

interfaces/mount: keep track of kept mount entries

5065bfa

This assists in computing the effective current profile as all the kept and mounted things are in in. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

zyga force-pushed the feature/update-ns/tool branch from 75b4d7f to 7e8309f Compare April 24, 2017 22:25

zyga force-pushed the feature/update-ns/tool branch from 7e8309f to 604c5fa Compare April 24, 2017 22:26

stolowski suggested changes Apr 25, 2017

View reviewed changes

zyga added 4 commits April 25, 2017 12:27

cmd/snap-update-ns: quit silently if there is no mount namespace

704536f

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: do nothing on both ENOENT and EINVAL

e326d18

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

tests: update snap-update-ns tests

a9ab60d

cmd/snap-update-ns: remove unneeded logging

6a0ef53

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

stolowski reviewed Apr 25, 2017

View reviewed changes

niemeyer previously requested changes Apr 25, 2017

View reviewed changes

zyga added 6 commits April 27, 2017 09:42

cmd/snap-update-ns: reword error message

4eaa394

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: tweak error messages to mention snap name

b6806a8

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: correct unlock/locking sequence

dda3dc2

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: use spearate variables for current-{before,after}

901c32b

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

Merge branch 'master' of github.com:snapcore/snapd into feature/updat…

eb3451d

…e-ns/tool

tests: adjust tests to check if shared content changes

844cfae

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

stolowski suggested changes Apr 28, 2017

View reviewed changes

stolowski approved these changes Apr 28, 2017

View reviewed changes

zyga added 4 commits May 3, 2017 09:43

Merge branch 'master' of github.com:snapcore/snapd into feature/updat…

a4d311c

…e-ns/tool

cmd/snap-update-ns: quit silently if there is no mount namespace

7c9c391

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: do nothing on both ENOENT and EINVAL

8bc7ef2

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

zyga added 14 commits May 15, 2017 13:01

tests: update snap-update-ns tests

578e410

cmd/snap-update-ns: remove unneeded logging

507dc65

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: reword error message

06e29bf

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: tweak error messages to mention snap name

0b81c19

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: correct unlock/locking sequence

e49d941

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: use spearate variables for current-{before,after}

8e94314

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

interfaces/mount: update snap namespace when setting up

2157637

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

tests: adjust tests to check if shared content changes

a33dfe2

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

tests: don't run update-ns manually, it is running automatically now

5bf9a38

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

Merge branch 'feature/update-ns/working' into feature/update-ns/tool

ccd922b

cmd/snap-update-ns: remove useless load of mountinfo

c824509

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

cmd/snap-update-ns: fix golint issues

ad062e3

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

chipaca reviewed May 15, 2017

View reviewed changes

zyga added 2 commits May 15, 2017 15:49

cmd/snap-update-ns: tweak variable name

17e40a8

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

tests: rework confusing comment

90f87f0

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

chipaca approved these changes May 15, 2017

View reviewed changes

zyga merged commit 295dfb6 into snapcore:master May 15, 2017

zyga deleted the feature/update-ns/tool branch May 15, 2017 15:17

cmd/snap-update-ns: add actual implementation #3225

cmd/snap-update-ns: add actual implementation #3225

Conversation

zyga commented Apr 24, 2017

zyga commented Apr 24, 2017

stolowski left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyga Apr 25, 2017 • edited

Choose a reason for hiding this comment

zyga commented Apr 25, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

niemeyer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stolowski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyga commented Apr 28, 2017

stolowski left a comment

Choose a reason for hiding this comment

chipaca left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stolowski left a comment •

edited

zyga Apr 25, 2017 •

edited

zyga commented Apr 25, 2017 •

edited