many: discard preserved namespace after removing snap #1847

Merged
merged 21 commits into from Sep 26, 2016

Conversation

Projects
None yet
4 participants
Contributor

zyga commented Sep 5, 2016

This branch adds support for new internal command, snap-discard-namespace, that is provided by the snap-confine project. The purpose of the command is to discard preserved namespaces so that removed snap is not holding kernel resources associated with any preserved namespaces.

zyga added some commits Sep 5, 2016

dirs: add LibExecDir variable
This variable holds the location of the libexecdir for snapd.
Usually it is /usr/lib/snapd/ but on some distributions it may be patched
to default to a different value.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
dirs: add InternalCmdPath
This patch adds a function for looking up the full path of an internal
snap command (e.g. snap-confine).

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
testutil: Add MockInternalCmdPath
This patch adds test function for mocking the lookup path of internal
commands. This allows them to be mocked with MockCommand

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
overlord/ifacestate: add "discard-namespace" task
This patch adds a new task to the interface manager. It can be used
to discard any persistent namespaces that may be held by a given snap.

It is meant to be used similarly to the existing discard-conns snap,
when a given snap is removed.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
overlord/snapstate: run "discard-namespace" when completely removing …
…snap

This patch makes the snap manager enqueue the "discard-namespace" task
when as snap is being entirely removed. This task runs exactly at the
same moment the existing "discard-conns" task would run.

This way preserved namespaces belonging to snaps that have been removed
are not dangling around and keeping kernel resources busy.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

@niemeyer niemeyer added the Blocked label Sep 5, 2016

Contributor

niemeyer commented Sep 5, 2016

Let's please only discuss this once RTM is golden.

@zyga zyga referenced this pull request in snapcore/snap-confine Sep 12, 2016

Merged

Add snap-discard-ns #135

dirs/dirs.go
@@ -91,6 +93,11 @@ func StripRootDir(dir string) string {
return "/" + result
}
+// InternalCmdPath returns the full path to a given internal command (e.g. to snap-confine)
+func InternalCmdPath(cmd string) string {
@mvo5

mvo5 Sep 14, 2016

Collaborator

I don't think we need this, we don't have this for any other for the dirs.

@zyga

zyga Sep 22, 2016

Contributor

Ack, I'll remove it

overlord/ifacestate/handlers.go
+ // command is not meant to always succeed because the namespace may not
+ // have been created yet in practice (e.g. the snap was removed before it
+ // was started since last reboot).
+ cmd := exec.Command(dirs.InternalCmdPath("snap-discard-ns"), snapName)
@mvo5

mvo5 Sep 14, 2016

Collaborator

filepath.Join(dirs.LibexecDir, "snap-discard-ns") seems to be ok here?

@zyga

zyga Sep 15, 2016

Contributor

Sure it is just more "magic" and less obvious what this means. I can change it if if you like though.

overlord/ifacestate/ifacemgr_test.go
+ si := s.mockSnap(c, sampleSnapYaml)
+
+ // Mock enough bits so that we can observe calls to snap-discard-ns
+ defer testutil.MockInternalCmdPath("")()
@mvo5

mvo5 Sep 14, 2016

Collaborator

I think this is slightly confusing, restorer := testutils.MockInternalCmdPath("")\ndefer restorer()\n would make it easier for me to read.

@mvo5

mvo5 Sep 14, 2016

Collaborator

I'm not sure we really need this MockInternalCmdPath (just yet), we could simply do "dirs.LibexecDir = cmd.BinDir` after the mockCommand was created.

testutil/internal.go
+
+// MockInternalCmdPath mocks the path used for lookups of internal commands,
+// such as snap-confine, that are not on PATH.
+func MockInternalCmdPath(path string) (restore func()) {
@mvo5

mvo5 Sep 14, 2016

Collaborator

See above, not sure we need this, given that we generally do something like "dirs.SetRootDir(c.MkDir())" in the tests, so this is usually already pointing to a place that we can mock around plus it gets restored for free usually.

Collaborator

mvo5 commented Sep 14, 2016

Code looks fine, I have some suggestion for simplification. I would love to hear what Gustavo has to say here before merging it, I'm not sure what he had in mind when he wrote the comment.

Also technically we require a database migration and need to add this task to existing changes. However given that this is currently a non-op (and that the corresponding binary does not even exists yet) that seems to be not needed.

zyga added some commits Sep 22, 2016

many: simplify mocking of snap-discard-ns
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Discard preserved namespaces around tests
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Contributor

zyga commented Sep 22, 2016

This failed only on create-key (pinging @mvo5 for review as this is urgent)

zyga added some commits Sep 22, 2016

tests: adjust test that is no longer valid with ns sharing
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
debain,tests: move discard ns logic to postrm script
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
debian/snapd.postrm
+ echo "Discarding preserved snap namespaces"
+ # opportunistic as those might not be actually mounted
+ for mnt in /run/snapd/ns/*.mnt; do
+ umount "$mnt" || true
@mvo5

mvo5 Sep 22, 2016

Collaborator

Would a umount --lazy make sense here?

@zyga

zyga Sep 22, 2016

Contributor

Yes, good catch

debian/snapd.postrm
+ for mnt in /run/snapd/ns/*.mnt; do
+ umount "$mnt" || true
+ done
+ umount /run/snapd/ns/ || true
@mvo5

mvo5 Sep 22, 2016

Collaborator

Same here? --lazy?

@zyga

zyga Sep 22, 2016

Contributor

This is a bind mount "over itself" perhaps we could actually leave it alone entirely. I'll use --lazy "just in case"

debian: use umount --lazy to clean up namespaces
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

Thanks. I've addressed the two comments.

debian/snapd.postrm
+ echo "Discarding preserved snap namespaces"
+ # opportunistic as those might not be actually mounted
+ for mnt in /run/snapd/ns/*.mnt; do
+ umount "$mnt" || true
@mvo5

mvo5 Sep 22, 2016

Collaborator

Would a umount --lazy make sense here?

@zyga

zyga Sep 22, 2016

Contributor

Yes, good catch

debian/snapd.postrm
+ for mnt in /run/snapd/ns/*.mnt; do
+ umount "$mnt" || true
+ done
+ umount /run/snapd/ns/ || true
@mvo5

mvo5 Sep 22, 2016

Collaborator

Same here? --lazy?

@zyga

zyga Sep 22, 2016

Contributor

This is a bind mount "over itself" perhaps we could actually leave it alone entirely. I'll use --lazy "just in case"

overlord/ifacestate/handlers.go
@@ -303,3 +306,24 @@ func (m *InterfaceManager) doDisconnect(task *state.Task, _ *tomb.Tomb) error {
setConns(st, conns)
return nil
}
+
+func (m *InterfaceManager) doDiscardNamespace(task *state.Task, _ *tomb.Tomb) error {
@niemeyer

niemeyer Sep 22, 2016

Contributor

Why is this in the interface manager? Isn't that a foundational property of every snap?

@zyga

zyga Sep 22, 2016

Contributor

To some extent yes but you can envision a sister task that lives right next to this one that goes into a namespace to reconfigure it based on interface changes. I think we can move it to the snapcore package but depending on how things unfold it might "go back" here.

@niemeyer

niemeyer Sep 22, 2016

Contributor

Interfaces can tweak it, but this sounds like a property of every snap. We have unlink-snap, unlink-current-snap, clear-snap, and discard-snap. Why do we need another task?

@zyga

zyga Sep 23, 2016

Contributor

You are right, it was just my knee-jerk reaction to put it in ifacestate and make it a task.

overlord/ifacestate/handlers.go
+
+ // Run "snap-discard-ns $SNAP_NAME" and ignore the error code. This
+ // command is not meant to always succeed because the namespace may not
+ // have been created yet in practice (e.g. the snap was removed before it
@niemeyer

niemeyer Sep 22, 2016

Contributor

That sounds bad. If we want to ignore a particular condition, we should teach snap-discard-ns to ignore that one particular condition, probably behind a flag. Or even better: why do we run this command if the namespace is not there? Simply ignoring the fact it fails in whichever way will hide real bugs. The tool might not even be there, and this will still pretend it's all good.

Let's please get both the error code and the combined output, and actually fail the task if it doesn't work. We want to know.

@zyga

zyga Sep 22, 2016

Contributor

I think you are right. I've patched snap-discard-ns since writing this code to be resilient to this error. I'll make sure we actually check for errors in this task.

overlord/ifacestate/ifacemgr.go
@@ -62,6 +62,7 @@ func Manager(s *state.State, extra []interfaces.Interface) (*InterfaceManager, e
runner.AddHandler("setup-profiles", m.doSetupProfiles, m.doRemoveProfiles)
runner.AddHandler("remove-profiles", m.doRemoveProfiles, m.doSetupProfiles)
runner.AddHandler("discard-conns", m.doDiscardConns, m.undoDiscardConns)
+ runner.AddHandler("discard-namespace", m.doDiscardNamespace, nil)
@niemeyer

niemeyer Sep 22, 2016

Contributor

Reading these changes it's unclear to me whether this command is about mount namespaces specifically, or other namespaces too. What's the case here? The task and the tool need to be named after their actual intended functionality, and it must be realistic going forward.

@niemeyer

niemeyer Sep 22, 2016

Contributor

In fact, do we even need a new task for that? Is there any benefit of doing this in isolation rather than in an existing removal task?

@niemeyer

niemeyer Sep 22, 2016

Contributor

Also, what about undos?

@zyga

zyga Sep 22, 2016

Contributor

Noted, I'll adjust it to be explicit about what is going on.

@niemeyer

niemeyer Sep 22, 2016

Contributor

Should probably be done inside discard-snap. That's where we remove the snap files, and there's no undo there either ATM (might change with trashed data + cleanup), so if we want to undo a discard we need to fix both cases.

@niemeyer

niemeyer Sep 22, 2016

Contributor

Implementation note: should discard the namespace after the snap file and the respective mount units are removed, as this is the point of no return.

@zyga

zyga Sep 22, 2016

Contributor

Ack, that's a good point, thank you!

This LGTM, beyond agreeing with gustavo's concerns about it not needing to be a task. And a minor nit.

debian/snapd.postrm
+ for mnt in /run/snapd/ns/*.mnt; do
+ umount --lazy "$mnt" || true
+ done
+ umount --lazy /run/snapd/ns/ || true
@chipaca

chipaca Sep 22, 2016

Member

can you use -l instead of --lazy? that'll keep @vosst happy

@zyga

zyga Sep 22, 2016

Contributor

Certainly, gosh, I should have known this :)

@mvo5

mvo5 Sep 26, 2016

Collaborator

Is this addressed yet?

@zyga

zyga Sep 26, 2016

Contributor

Done, thanks, I missed this earlier. I changed all occurrences of umount --lazy to umount -l

overlord/ifacestate: handle errors from snap-discard-ns
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

@mvo5 mvo5 added Critical and removed Blocked labels Sep 23, 2016

zyga added some commits Sep 26, 2016

overlord: move discard-ns to snapstate
This patch moves the responsibility of discarding a snap namespace from
ifacestate to snapstate fuses it with the discard-snap task so that
separate task is not necessary.
overlord/snapstate/backend: handle old snap-confine
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
overlord: mock snap-discard-ns for testing
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
tests: adjust securiry-private-tmp test to support old snap-confine
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Contributor

zyga commented Sep 26, 2016

Tests green! Please review me

Collaborator

mvo5 commented Sep 26, 2016

Looks nice, thank you! Two small comments.

mvo5 approved these changes Sep 26, 2016

Looks good, assuming my two comments get addressed.

debian/snapd.postrm
+ for mnt in /run/snapd/ns/*.mnt; do
+ umount --lazy "$mnt" || true
+ done
+ umount --lazy /run/snapd/ns/ || true
@chipaca

chipaca Sep 22, 2016

Member

can you use -l instead of --lazy? that'll keep @vosst happy

@zyga

zyga Sep 22, 2016

Contributor

Certainly, gosh, I should have known this :)

@mvo5

mvo5 Sep 26, 2016

Collaborator

Is this addressed yet?

@zyga

zyga Sep 26, 2016

Contributor

Done, thanks, I missed this earlier. I changed all occurrences of umount --lazy to umount -l

overlord/snapstate/backend/ns.go
+ cmd := exec.Command(snapDiscardNs, snapName)
+ output, err := cmd.CombinedOutput()
+ if err != nil {
+ return fmt.Errorf("cannot discard preserved namespaces of snap %q: %s", snapName, output)
@mvo5

mvo5 Sep 26, 2016

Collaborator

We probably want osutil.OutputErr() here.

@zyga

zyga Sep 26, 2016

Contributor

Done, thanks, I didn't know about this

zyga added some commits Sep 26, 2016

overlord/snapstate/backend: use OutputErr
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
debian: use unmount -l instead of --lazy
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

One question, and LGTM.

overlord/snapstate/backend/ns.go
+ if osutil.FileExists(snapDiscardNs) {
+ // Preserved namespaces need to be discarded only if they are being
+ // preserved by snap-confine. If the snap-discard-ns command doesn't
+ // exist then the shared mount namespace feature is not available and
@niemeyer

niemeyer Sep 26, 2016

Contributor

This is not quite a safe assumption. Can't we test for the existence of the mount namespace file instead?

@zyga

zyga Sep 26, 2016

Contributor

Totally correct, done!

zyga added some commits Sep 26, 2016

dirs: add variable for /run/snapd/ns
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
overlord/snapstate/backend: detect presence of .mnt files
This patch tweaks the use of snap-discard-ns. Instead of detecting if
the command is available or not we are now detecting if the .mnt file
exists. This will make snapd fail loudly if something changes in
snap-confine without the corresponding change in snapd.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

@zyga zyga merged commit 1850e8f into snapcore:master Sep 26, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@zyga zyga deleted the zyga:discard-namespace branch Sep 26, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment