Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] many: export tools from core/snapd to mount namespaces #8843

Closed
wants to merge 18 commits into from

Conversation

zyga
Copy link
Collaborator

@zyga zyga commented Jun 9, 2020

This branch contains a quick and dirty implementation of the approach "C" of
https://forum.snapcraft.io/t/12139 - which allows us to update snapd or core
snap freely and keeping updated tools visible to running snaps.

I did not add any tests, but if you run spread you can confirm that this is
indeed working as expected. The code in the new export manager is extremely
simplistic, and in a proper implementation would need to handle many more
cases.

The branch has some logical progression through subsequent patches but you can
see it as the following conceptual list:

  • A snap can export content to the host, in a special location that nothing
    currently looks at. This way classic world needs to be adapted to look for
    the new content, with full consequences. I picked the location
    /var/lib/snapd/export/ as the root of exported files. Files can be exported,
    in this implementation, only as symlinks, although other options are possible
    for discussion.

  • Similar to implicit slots, both the core snap and the snapd snap gain
    implicit export man that exposes their /usr/lib/snapd directory, as
    core/tools and snapd/tools respectively. Note that the exported directory is
    named after a snap (core and snapd) but in general that is not required. For
    example, many snaps could export fonts to fonts/FONT_NAME or manual pages to
    man/manN/PAGE.N.

  • The new export manager handles the new pair of tasks, export-content and
    unexport-content, which are sequenced immediately after link-snap and before
    unlink-snap. Conceptually the export manager should handle the state of what
    is exported from which snaps and how conflicts are resolved. In this naive
    implementation there are no conflicts so everything is just exported at all
    times.

  • Lastly snap-confine is modified to bind mount a read-only view of
    /var/lib/snapd/exports/{snapd,core}/tools/ as /usr/lib/snapd/ so that snaps
    using core18 or core20 as base can get a copy of things like snap-exec and
    snapctl. The whole benefit comes from the fact that the tools directory doesn't
    need to be changed via the mount system anymore and that the export manager will
    rewrite the symlinks placed inside. So for example tools/snap-exec will point to
    /snap/snapd/$SNAP_REVISION/usr/lib/snapd/snap-exec - with the full path being
    visible inside the mount namespace.

@pedronis pedronis self-requested a review June 9, 2020 18:10
@pedronis pedronis added the Needs Samuele review Needs a review from Samuele before it can land label Jun 9, 2020
zyga added 18 commits June 10, 2020 09:40
Snaps can export content to the host. Exports are described by a map of
paths, as seen from a fixed location on the host, to files and
directories relative to the mount point of a specific snap revision.

Currently exports cannot be declared by YAML and are reserved for the
snapd snap.

Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
The snapd.snap and core.snap declare implicit exports specific files
from internal tools directory "$SNAP/usr/lib/snapd/$file" as
"/var/lib/snapd/export/{core,snapd}/tools/$file"

This will later on be picked up by snap-confine, as the source of the
internal tools. Note that this design allows mounting "tools" and not
having to perform any mount operations for other updates.

Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
The export manager is responsible for exporting content from snaps to a
designated location on the host. The host software must actively reach
out to look for the exported content, so exporting content is safe as
the burden is shifted on the party performing the integration.

Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Only the symlink method is currently implemented. The symbolic link uses
absolute path to /snap/$SNAP_REVISION/$SNAP_NAME/$private_name, so that
it works inside the mount namespace.

Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Bulk of the change is the incredibly duplicated and fragile test changes
to accommodate the new task in the chain.

Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
@zyga
Copy link
Collaborator Author

zyga commented Jul 8, 2020

I'm doing a second pass on this idea with a slightly different details. My plan is to do the following:

The tools from all sources are going to be exported to distinct sub-directories of /var/lib/snapd/export. This design puts all the tools on a plate (from each revision of snaps and from the host) and allows us to atomically switch between them through filesystem manipulation. I will write the details later but the idea allows us to perform atomic and robust "update" that ensures that at no point tools are unavailable.

I'm working on a spike-from-a-spike to get a proof of concept working.

@zyga
Copy link
Collaborator Author

zyga commented Sep 1, 2020

This branch is superseded by a new approach I'm implementing and proposing parts of separately. I will still use some of the code that is present here but as far as the RFC was concerned, it has done its job.

@zyga zyga closed this Sep 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Samuele review Needs a review from Samuele before it can land
Projects
None yet
2 participants