New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with the snap installed version #3143

Closed
aquilax opened this Issue Mar 8, 2017 · 13 comments

Comments

Projects
None yet
5 participants
@aquilax

aquilax commented Mar 8, 2017

Trying to use hugo installed through snap on Ubuntu 16.04.1 LTS, I get the following error:

aquilax@hp:/tmp/htest$ hugo new site . -v
runtime/cgo: pthread_create failed: Resource temporarily unavailable
Error: /var/lib/snapd/void already exists and is not empty

When I provide the full path, I get Congratulations but the directory remains empty:

tmp/htest$ hugo new site /tmp/htest/ -v
Congratulations! Your new Hugo site is created in /tmp/htest.

Just a few more steps and you're ready to go:

1. Download a theme into the same-named folder.
   Choose a theme from https://themes.gohugo.io/, or
   create your own with the "hugo new theme <THEMENAME>" command.
2. Perhaps you want to add some content. You can add single files
   with "hugo new <SECTIONNAME>/<FILENAME>.<FORMAT>".
3. Start the built-in live server via "hugo server".

Visit https://gohugo.io/ for quickstart guide and full documentation.
aquilax@hp:/tmp/htest$ ls -la
total 28
drwxrwxr-x  2 aquilax aquilax  4096 Mar  8 08:12 .
drwxrwxrwt 45 root    root    24576 Mar  8 08:15 ..
@bep

This comment has been minimized.

Show comment
Hide comment
@bep
Member

bep commented Mar 8, 2017

@anthonyfok

This comment has been minimized.

Show comment
Hide comment
@anthonyfok

anthonyfok Mar 8, 2017

Contributor

@bep, thank you for the /cc!

Hi @aquilax,

What you are seeing, as inconvenient it may be, is Snaps' default and expected behaviour for security reasons. To quote https://snapcraft.io/:


Snaps are read-only and have segregated data stores

Snaps are read-only for security. We want to prevent a hostile party from sneakily changing the software on your machine, so you cannot modify a snap that is installed on your system. This also means you can always check the signature on the snap, even long after you installed it, to make sure it is still exactly the software you intended. If you want to modify a snap, you can usually build your own version of it, especially if it is open source.

So where can a snap write data? Each snap gets its own set of writable directories which have specific properties. There are two directories which the snap can write to independent of the user. One of these is versioned - each time the snap is upgraded the data is saved and the new snap revision can upgrade its copy. The other ‘common’ data directory is not versioned and is used for big blobs of data that you don’t want to duplicate across revisions of the snap:

/var/snap/<name>/current/  ← $SNAP_DATA is the versioned snap data directory
/var/snap/<name>/common/   ← $SNAP_COMMON will not be versioned on upgrades

Typically, configuration is stored in one of these, along with system-wide data for the snap.

There are also an equivalent two writable directories for each snap in the user home, which can be used to store snap data that is specific to one user or another, separately:

~/snap/<name>/current/      ← $SNAP_USER_DATA that can be rolled back
~/snap/<name>/common/       ← $SNAP_USER_COMMON unversioned user-specific data

This way applications can be immutable for security reasons while still offering a full and rich user experience. To learn more about the makeup of a snap please see the snap architecture article.


So, for most snaps, non-root users may only write inside two specific directories, and for Hugo, I would have had to limited myself to ~/snap/hugo/common/.

Thankfully, in the snapcraft.yaml "recipe" kindly provided by our friends like @dholbach from Canonical, this restriction was relaxed with the use of the home plug which allows the user to "access non-hidden files in user's $HOME and gvfs mounted directories owned by the user to read/write/lock." (See https://snapcraft.io/docs/reference/interfaces).

What it means is that you can use the snap edition of Hugo to write (i.e. run hugo new site etc.) anywhere inside your $HOME directory, say ~/tmp/htest, but not in the system-wide /tmp directory.

Should we relax this restriction even further? I suppose we could (I don't know how yet), but since that would go against Snaps' security model and design philosophy, so I have my reservations too.

So, perhaps the "write inside $HOME only" needs to be documented, perhaps in docs/content/overview/installing.md right after the $ snap install hugo command?

Contributor

anthonyfok commented Mar 8, 2017

@bep, thank you for the /cc!

Hi @aquilax,

What you are seeing, as inconvenient it may be, is Snaps' default and expected behaviour for security reasons. To quote https://snapcraft.io/:


Snaps are read-only and have segregated data stores

Snaps are read-only for security. We want to prevent a hostile party from sneakily changing the software on your machine, so you cannot modify a snap that is installed on your system. This also means you can always check the signature on the snap, even long after you installed it, to make sure it is still exactly the software you intended. If you want to modify a snap, you can usually build your own version of it, especially if it is open source.

So where can a snap write data? Each snap gets its own set of writable directories which have specific properties. There are two directories which the snap can write to independent of the user. One of these is versioned - each time the snap is upgraded the data is saved and the new snap revision can upgrade its copy. The other ‘common’ data directory is not versioned and is used for big blobs of data that you don’t want to duplicate across revisions of the snap:

/var/snap/<name>/current/  ← $SNAP_DATA is the versioned snap data directory
/var/snap/<name>/common/   ← $SNAP_COMMON will not be versioned on upgrades

Typically, configuration is stored in one of these, along with system-wide data for the snap.

There are also an equivalent two writable directories for each snap in the user home, which can be used to store snap data that is specific to one user or another, separately:

~/snap/<name>/current/      ← $SNAP_USER_DATA that can be rolled back
~/snap/<name>/common/       ← $SNAP_USER_COMMON unversioned user-specific data

This way applications can be immutable for security reasons while still offering a full and rich user experience. To learn more about the makeup of a snap please see the snap architecture article.


So, for most snaps, non-root users may only write inside two specific directories, and for Hugo, I would have had to limited myself to ~/snap/hugo/common/.

Thankfully, in the snapcraft.yaml "recipe" kindly provided by our friends like @dholbach from Canonical, this restriction was relaxed with the use of the home plug which allows the user to "access non-hidden files in user's $HOME and gvfs mounted directories owned by the user to read/write/lock." (See https://snapcraft.io/docs/reference/interfaces).

What it means is that you can use the snap edition of Hugo to write (i.e. run hugo new site etc.) anywhere inside your $HOME directory, say ~/tmp/htest, but not in the system-wide /tmp directory.

Should we relax this restriction even further? I suppose we could (I don't know how yet), but since that would go against Snaps' security model and design philosophy, so I have my reservations too.

So, perhaps the "write inside $HOME only" needs to be documented, perhaps in docs/content/overview/installing.md right after the $ snap install hugo command?

@anthonyfok anthonyfok added the Docs label Mar 8, 2017

@aquilax

This comment has been minimized.

Show comment
Hide comment
@aquilax

aquilax Mar 8, 2017

Thank you for the detailed response, @anthonyfok.

I ended building hugo from source, but for non-technical user, trying snap, that restriction could be confusing, since the tool itself also does not provide any user friendly error message.

Documenting this limitation in the documentation would be definite improvement.

aquilax commented Mar 8, 2017

Thank you for the detailed response, @anthonyfok.

I ended building hugo from source, but for non-technical user, trying snap, that restriction could be confusing, since the tool itself also does not provide any user friendly error message.

Documenting this limitation in the documentation would be definite improvement.

@rdwatters

This comment has been minimized.

Show comment
Hide comment
@rdwatters

rdwatters Mar 8, 2017

Contributor

@anthonyfok I'd like your input on whether this is something that should be added to upcoming documentation for Linux installation (not in its entirety, but maybe as a callout/admonition => just 1 or 2 sentences).

In other words, is this a common "gotcha" for Linux users?

Contributor

rdwatters commented Mar 8, 2017

@anthonyfok I'd like your input on whether this is something that should be added to upcoming documentation for Linux installation (not in its entirety, but maybe as a callout/admonition => just 1 or 2 sentences).

In other words, is this a common "gotcha" for Linux users?

@anthonyfok

This comment has been minimized.

Show comment
Hide comment
@anthonyfok

anthonyfok Mar 8, 2017

Contributor

Hi @rdwatters,

Thank you for asking! Yes, this is something that needed to be added to the Hugo documentation, maybe for both installation instruction and for the hugo new site command, and I agree, 1 or 2 sentences would suffice, probably with a link pointing to this GitHub issue. :-)

As in whether it is a common "gotcha"? Yes, it is for all Linux users who decided to install Hugo as a "snap", but such restriction does not exist for .deb, .rpm or .tar.gz packages/archives.

My personal favourite is actually apt-get install hugo (for downloading .deb from Debian or Ubuntu), except that it takes more work, thus more days, to keep it up-to-date, hence while the .deb package is still at 0.18.1, the Snap packages is already at 0.19. Plus the fact that Hugo 0.18.1 or 0.19 .deb package is probably never going to trickle back to Ubuntu 16.04.1 LTS (Long-term support), the Hugo Snap becomes a really attractive alternative.

That said, while Snaps are universal and do work on Fedora and other non-Debian/non-Ubuntu Linux distributions, I would wildly guess that 90% of Hugo Snap users are Ubuntu users. Mind you, Ubuntu is the most popular Linux distribution, so I foresee more and more users running into this "write in $HOME only" restriction.

(The Fedora/Red Hat/GNOME world has their own Flatpak which is somewhat similar to Snaps, though I haven't used Flatpak and thus cannot comment on its similar "write in $HOME only" restriction, if any.)

Contributor

anthonyfok commented Mar 8, 2017

Hi @rdwatters,

Thank you for asking! Yes, this is something that needed to be added to the Hugo documentation, maybe for both installation instruction and for the hugo new site command, and I agree, 1 or 2 sentences would suffice, probably with a link pointing to this GitHub issue. :-)

As in whether it is a common "gotcha"? Yes, it is for all Linux users who decided to install Hugo as a "snap", but such restriction does not exist for .deb, .rpm or .tar.gz packages/archives.

My personal favourite is actually apt-get install hugo (for downloading .deb from Debian or Ubuntu), except that it takes more work, thus more days, to keep it up-to-date, hence while the .deb package is still at 0.18.1, the Snap packages is already at 0.19. Plus the fact that Hugo 0.18.1 or 0.19 .deb package is probably never going to trickle back to Ubuntu 16.04.1 LTS (Long-term support), the Hugo Snap becomes a really attractive alternative.

That said, while Snaps are universal and do work on Fedora and other non-Debian/non-Ubuntu Linux distributions, I would wildly guess that 90% of Hugo Snap users are Ubuntu users. Mind you, Ubuntu is the most popular Linux distribution, so I foresee more and more users running into this "write in $HOME only" restriction.

(The Fedora/Red Hat/GNOME world has their own Flatpak which is somewhat similar to Snaps, though I haven't used Flatpak and thus cannot comment on its similar "write in $HOME only" restriction, if any.)

@rdwatters

This comment has been minimized.

Show comment
Hide comment
@rdwatters

rdwatters Mar 10, 2017

Contributor

@anthonyfok I appreciate the patience and feedback. 😄 Is this sufficient?

Contributor

rdwatters commented Mar 10, 2017

@anthonyfok I appreciate the patience and feedback. 😄 Is this sufficient?

@rdwatters

This comment has been minimized.

Show comment
Hide comment
@rdwatters

rdwatters Mar 11, 2017

Contributor

Oh, and @aquilax, your input is welcome too. Thanks guys.

Contributor

rdwatters commented Mar 11, 2017

Oh, and @aquilax, your input is welcome too. Thanks guys.

@aquilax

This comment has been minimized.

Show comment
Hide comment
@aquilax

aquilax Mar 11, 2017

@rdwatters Looks much better than before.
I'm not sure about the You can find a workaround using the home plug at snapcraft.io sentence only. I see list of interfaces on the page but nothing that suggests a workaround there.

aquilax commented Mar 11, 2017

@rdwatters Looks much better than before.
I'm not sure about the You can find a workaround using the home plug at snapcraft.io sentence only. I see list of interfaces on the page but nothing that suggests a workaround there.

@anthonyfok

This comment has been minimized.

Show comment
Hide comment
@anthonyfok

anthonyfok Mar 13, 2017

Contributor

Hi @rdwatters!

Thank you for updating the documentation! It looks really nice, though the so-called "home plug" workaround is actually for us Hugo developers who maintain the snapcraft.yaml recipe. Hugo Snaps users have no way of changing it. The Hugo snap package has used the home plug since day one, and users of this Hugo snap has always been able to write inside the user's own home directory, just not outside of it.

I just discovered something new though: It turns out there is a new classic confinement (versus the default strict confinement) that was added in December 2016:

To quote:

A snap in classic confinement behaves as a traditionally packaged application, with full access to the system. As opposed to strict and devmode, what a classic snap sees as “/” is the host system’s “/” and not the core snap’s “/”. Snaps using this fully open security policy are manually reviewed in the store and are only allowed on systems where snapd is installed on top of a traditional Linux distribution, as opposed to system booting from an Ubuntu Core image. They can be released in all store channels.

So, it is possible, but users would then have to type snap install hugo --classic, and they can’t use it in a "Ubuntu Core image". And yes, it somewhat defeats Snap’s original security confinement model too.


Anyhow, back to the actual documentation text. :-)

I recommend making the following revisions:

Installing From Snap Hugo as a Snap Package

In any of the Linux distributions that support snaps:

snap install hugo

Snap Hugo users may get a Resource temporarily unavailable runtime error. This is because non-root users may only write inside two specific directories for most snaps. You can find a workaround using the home plug at snapcraft.io.
Due to snaps’ confinement and security model, Hugo-as-a-snap can write only inside the user’s $HOME directory (and gvfs mounted directories owned by the user). More information is also available in this related GitHub issue.

Also, I notice the absence of installing Hugo as a traditional .deb package using apt. Please add the following section too, which, incidentally, is my favourite method of installing Hugo:

Debian/Ubuntu-based distributions

Hugo has been included in Debian and Ubuntu since 2016, and thus installing Hugo is as simple as:

sudo apt install hugo

Pros:

  • Native Debian/Ubuntu package maintained by Debian Developers
  • Pre-installed bash completion script and man pages for best interactive experience

Cons:

  • Might not be the latest version, especially if you are using an older stable version like Ubuntu 16.04 LTS. Until backports and PPA are available, you may consider installing the Hugo snap package to get the latest version of Hugo, as described below.

Yes, I recommend placing the secions on Arch and Debian/Ubuntu together, and placing the Snap section right underneath. ;-)

By the way, English is not my first language, so please feel free to revise whatever I have written.

Many thanks @rdwatters!

Contributor

anthonyfok commented Mar 13, 2017

Hi @rdwatters!

Thank you for updating the documentation! It looks really nice, though the so-called "home plug" workaround is actually for us Hugo developers who maintain the snapcraft.yaml recipe. Hugo Snaps users have no way of changing it. The Hugo snap package has used the home plug since day one, and users of this Hugo snap has always been able to write inside the user's own home directory, just not outside of it.

I just discovered something new though: It turns out there is a new classic confinement (versus the default strict confinement) that was added in December 2016:

To quote:

A snap in classic confinement behaves as a traditionally packaged application, with full access to the system. As opposed to strict and devmode, what a classic snap sees as “/” is the host system’s “/” and not the core snap’s “/”. Snaps using this fully open security policy are manually reviewed in the store and are only allowed on systems where snapd is installed on top of a traditional Linux distribution, as opposed to system booting from an Ubuntu Core image. They can be released in all store channels.

So, it is possible, but users would then have to type snap install hugo --classic, and they can’t use it in a "Ubuntu Core image". And yes, it somewhat defeats Snap’s original security confinement model too.


Anyhow, back to the actual documentation text. :-)

I recommend making the following revisions:

Installing From Snap Hugo as a Snap Package

In any of the Linux distributions that support snaps:

snap install hugo

Snap Hugo users may get a Resource temporarily unavailable runtime error. This is because non-root users may only write inside two specific directories for most snaps. You can find a workaround using the home plug at snapcraft.io.
Due to snaps’ confinement and security model, Hugo-as-a-snap can write only inside the user’s $HOME directory (and gvfs mounted directories owned by the user). More information is also available in this related GitHub issue.

Also, I notice the absence of installing Hugo as a traditional .deb package using apt. Please add the following section too, which, incidentally, is my favourite method of installing Hugo:

Debian/Ubuntu-based distributions

Hugo has been included in Debian and Ubuntu since 2016, and thus installing Hugo is as simple as:

sudo apt install hugo

Pros:

  • Native Debian/Ubuntu package maintained by Debian Developers
  • Pre-installed bash completion script and man pages for best interactive experience

Cons:

  • Might not be the latest version, especially if you are using an older stable version like Ubuntu 16.04 LTS. Until backports and PPA are available, you may consider installing the Hugo snap package to get the latest version of Hugo, as described below.

Yes, I recommend placing the secions on Arch and Debian/Ubuntu together, and placing the Snap section right underneath. ;-)

By the way, English is not my first language, so please feel free to revise whatever I have written.

Many thanks @rdwatters!

@rdwatters

This comment has been minimized.

Show comment
Hide comment
@rdwatters

rdwatters Mar 13, 2017

Contributor

@anthonyfok I've updated per all your recommendations. I appreciate your always thoughtful and well-articulated feedback. I've certainly learned a lot in this process. Also...

By the way, English is not my first language, so please feel free to revise whatever I have written.

A. I never would have guessed. Your English is fantastic!
B. Think of it this way: your skill level with the English language is 100 × that of our current president 😉

Thank you @aquilax as well. Cheers.

Contributor

rdwatters commented Mar 13, 2017

@anthonyfok I've updated per all your recommendations. I appreciate your always thoughtful and well-articulated feedback. I've certainly learned a lot in this process. Also...

By the way, English is not my first language, so please feel free to revise whatever I have written.

A. I never would have guessed. Your English is fantastic!
B. Think of it this way: your skill level with the English language is 100 × that of our current president 😉

Thank you @aquilax as well. Cheers.

@anthonyfok

This comment has been minimized.

Show comment
Hide comment
@anthonyfok

anthonyfok Mar 13, 2017

Contributor

Thank you @rdwatters!

@anthonyfok I've updated per all your recommendations. I appreciate your always thoughtful and well-articulated feedback. I've certainly learned a lot in this process.

Wow, that was quick! Thank you for fixing the docs so quickly!

Yes, I recommend placing the secions on Arch and Debian/Ubuntu together

Actually, what I meant to say is to place the Arch section and Debian/Ubuntu section one after the other. My bad for a poor choice of word "together". I am also hoping someone will step up with the RPM maintenance for the Fedora / Red Hat / openSUSE ecosystems too so we can add that in the near future. (Looks like someone has! https://discuss.gohugo.io/t/solved-fedora-copr-repository-out-of-service/2491/12)

That said, if you personally prefer keeping these distros all under one section, that is fine with me too. :-)

One more nitpicking:

Hugo-as-a-snap can write only inside the user’s $HOMEdirectory

Please add a missing space between "$HOME" and "directory":

Hugo-as-a-snap can write only inside the user’s $HOME directory

Many thanks!

Contributor

anthonyfok commented Mar 13, 2017

Thank you @rdwatters!

@anthonyfok I've updated per all your recommendations. I appreciate your always thoughtful and well-articulated feedback. I've certainly learned a lot in this process.

Wow, that was quick! Thank you for fixing the docs so quickly!

Yes, I recommend placing the secions on Arch and Debian/Ubuntu together

Actually, what I meant to say is to place the Arch section and Debian/Ubuntu section one after the other. My bad for a poor choice of word "together". I am also hoping someone will step up with the RPM maintenance for the Fedora / Red Hat / openSUSE ecosystems too so we can add that in the near future. (Looks like someone has! https://discuss.gohugo.io/t/solved-fedora-copr-repository-out-of-service/2491/12)

That said, if you personally prefer keeping these distros all under one section, that is fine with me too. :-)

One more nitpicking:

Hugo-as-a-snap can write only inside the user’s $HOMEdirectory

Please add a missing space between "$HOME" and "directory":

Hugo-as-a-snap can write only inside the user’s $HOME directory

Many thanks!

@rdwatters

This comment has been minimized.

Show comment
Hide comment
@rdwatters

rdwatters Mar 13, 2017

Contributor

Actually, what I meant to say is to place the Arch section and Debian/Ubuntu section one after the other.

I suspected that might be the case. Nevertheless, fixed.

Please add a missing space between "$HOME" and "directory":

Ah, keen eye. Thanks!

Contributor

rdwatters commented Mar 13, 2017

Actually, what I meant to say is to place the Arch section and Debian/Ubuntu section one after the other.

I suspected that might be the case. Nevertheless, fixed.

Please add a missing space between "$HOME" and "directory":

Ah, keen eye. Thanks!

@Tails

This comment has been minimized.

Show comment
Hide comment
@Tails

Tails Dec 12, 2017

When will the --classic flag be added to the docs? I had to find this issue in order for hugo to be able to write to a mounted partition in my $HOME.

Tails commented Dec 12, 2017

When will the --classic flag be added to the docs? I had to find this issue in order for hugo to be able to write to a mounted partition in my $HOME.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment