Support persistent paths via symlinks the same way bind-dirs does it via bind mounts

[ypid]

Currently, bind-dirs supports making paths persistent by bind mounting them from /rw. Depending on the situation, using symlinks instead of bind mounting might be better because programs generally have better support for them in my experience than bind mounted files in particular.

Ref: QubesOS/qubes-doc#299

My intention behind this is to provide a clean interface that configuration management systems and users can use to make certain configuration persistent. As I heavily use Ansible and DebOps for managing servers and also some parts of my Qubes OS workstation so I am working on making the Ansible roles I need support Qubes OS out of the box.

Ref: debops/ansible-persistent_paths#4

Does persistent paths via symlinks sound like worth having also for others?

If so, I see a few ways to implement this:

1. A new, unified bind-dirs script, maybe renamed into persistent-paths since it supports directories/files via bind mounts or symlinks (or potentially others, maybe copy which just copies files from /rw each time). I would also propose to unify /rw/bind-dirs/ into /rw/persistent-paths/ for this so that whether bind mounts, symlinks or something else is used can easily be switched via configuration. The rename to /rw/persistent-paths/ could be done automatically by persistent-paths.
2. Create a new script symlink-paths with a similar design as bind-dirs which would use an independent persistent storage like /rw/symlink-paths/.
3. Implement support for symlinks in an Ansible role debops.persistent_paths to provides this feature without a lower level interface on Qubes OS.

[marmarek]

Does persistent paths via symlinks sound like worth having also for others?

The problem with symlink approach, is that if some tool doesn't handle it correctly (and replace symlink with a plain file), you will silently loose that file at VM restart. Also, symlinks cause some problems when application try to get "real" path to the file/dir - for example when /home was a symlink to /rw/home, a lot of applications didn't correctly detected it (like bash not displaying ~ etc).
For some cases it may not be a problem (because you know your tools don't do that), but for general case I wouldn't recommend it.

A new, unified bind-dirs script, maybe renamed into persistent-paths since it supports directories/files via bind mounts or symlinks (or potentially others, maybe copy which just copies files from /rw each time). I would also propose to unify /rw/bind-dirs/ into /rw/persistent-paths/ for this so that whether bind mounts, symlinks or something else is used can easily be switched via configuration. The rename to /rw/persistent-paths/ could be done automatically by persistent-paths.

If adding this to Qubes OS, extending existing bind-dirs seems like a good idea, because a lot of it would be reused (most/all the handling of initial copy). But for the reason above, I wouldn't replace existing functionality. Instead, maybe add a second bash array handled by it (in addition to binds, have symlinks or such).

As for renaming, I'd avoid that even if the current name isn't 100% accurate. Mostly because it's already used in a lot of places, especially in Whonix, and renaming it would cause a lot of troubles. Cc: @adrelanos

[marmarek]

Does persistent paths via symlinks sound like worth having also for others?

The problem with symlink approach, is that if some tool doesn't handle it correctly (and replace symlink with a plain file), you will silently loose that file at VM restart. Also, symlinks cause some problems when application try to get "real" path to the file/dir - for example when /home was a symlink to /rw/home, a lot of applications didn't correctly detected it (like bash not displaying ~ etc).
For some cases it may not be a problem (because you know your tools don't do that), but for general case I wouldn't recommend it.

A new, unified bind-dirs script, maybe renamed into persistent-paths since it supports directories/files via bind mounts or symlinks (or potentially others, maybe copy which just copies files from /rw each time). I would also propose to unify /rw/bind-dirs/ into /rw/persistent-paths/ for this so that whether bind mounts, symlinks or something else is used can easily be switched via configuration. The rename to /rw/persistent-paths/ could be done automatically by persistent-paths.

If adding this to Qubes OS, extending existing bind-dirs seems like a good idea, because a lot of it would be reused (most/all the handling of initial copy). But for the reason above, I wouldn't replace existing functionality. Instead, maybe add a second bash array handled by it (in addition to binds, have symlinks or such).

As for renaming, I'd avoid that even if the current name isn't 100% accurate. Mostly because it's already used in a lot of places, especially in Whonix, and renaming it would cause a lot of troubles. Cc: @adrelanos

[adrelanos]

In past there was an AppArmor issue with symlinks rather than bind mount. It had been ported to symlinks then. Reference:

#1122

---

Should you rename the script, please keep an existing file /usr/lib/qubes/init/bind-dirs.sh (empty is okay) for Whonix compatibility.

if [ -e /usr/lib/qubes/bind-dirs.sh ] || [ -e /usr/lib/qubes/init/bind-dirs.sh ] ; then

(I am very much looking forward to purge the the compatibility stuff, but it requires Qubes R3.2 to be in status "deprecated / unsupported / Whonix upgrades will break it".)

---

Perhaps we'll keep variable binds as is an introduce another similar one symlinks?

I see no reason to make it either binds or symlinks? Why not support both? bind-dirs outside of Whonix is a pretty advanced technique from a usability point of view. One who wanted to add either binds or symlinks I guess will be able to figure out by reading the documentation which one suits the use case best.

Adding the functionality to the existing bind-dirs script seems like a good idea indeed, yes. Perhaps you can somehow refactor the code. Refactoring for fso_ro in "${binds[@]}"; do, perhaps by introducing one or two new bash functions.

[adrelanos]

In past there was an AppArmor issue with symlinks rather than bind mount. It had been ported to symlinks then. Reference:

#1122

---

Should you rename the script, please keep an existing file /usr/lib/qubes/init/bind-dirs.sh (empty is okay) for Whonix compatibility.

if [ -e /usr/lib/qubes/bind-dirs.sh ] || [ -e /usr/lib/qubes/init/bind-dirs.sh ] ; then

(I am very much looking forward to purge the the compatibility stuff, but it requires Qubes R3.2 to be in status "deprecated / unsupported / Whonix upgrades will break it".)

---

Perhaps we'll keep variable binds as is an introduce another similar one symlinks?

I see no reason to make it either binds or symlinks? Why not support both? bind-dirs outside of Whonix is a pretty advanced technique from a usability point of view. One who wanted to add either binds or symlinks I guess will be able to figure out by reading the documentation which one suits the use case best.

Adding the functionality to the existing bind-dirs script seems like a good idea indeed, yes. Perhaps you can somehow refactor the code. Refactoring for fso_ro in "${binds[@]}"; do, perhaps by introducing one or two new bash functions.

[Rudd-O]

Supporting both would be great. Turns out having bind-dirs skip bind-mounting onto stuff that doesn't exist upon boot of the AppVM, really sucks for automation. Now I have to remember to create the files / dirs in the TemplateVM, which really fucks up my automation config.

[Rudd-O]

Supporting both would be great. Turns out having bind-dirs skip bind-mounting onto stuff that doesn't exist upon boot of the AppVM, really sucks for automation. Now I have to remember to create the files / dirs in the TemplateVM, which really fucks up my automation config.

[ypid]

@Rudd-O Seen QubesOS/qubes-core-agent-linux#42 ?

[ypid]

@Rudd-O Seen QubesOS/qubes-core-agent-linux#42 ?

[Rudd-O]

On 03/20/2017 07:43 PM, Robin Schneider wrote: @Rudd-O <https://github.com/Rudd-O> Seen QubesOS/qubes-core-agent-linux#42 <QubesOS/qubes-core-agent-linux#42> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2714 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVIwjuxzdIu_jo2lM7dTIoMYKjlcYUaks5rntbegaJpZM4Mhq5C>.

Haven't pulled the updates yet. Will try as soon as it is available as a package.

…

-- Rudd-O http://rudd-o.com/

[Rudd-O]

On 03/20/2017 07:43 PM, Robin Schneider wrote: @Rudd-O <https://github.com/Rudd-O> Seen QubesOS/qubes-core-agent-linux#42 <QubesOS/qubes-core-agent-linux#42> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2714 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVIwjuxzdIu_jo2lM7dTIoMYKjlcYUaks5rntbegaJpZM4Mhq5C>.

Haven't pulled the updates yet. Will try as soon as it is available as a package.

…

-- Rudd-O http://rudd-o.com/

[ypid]

But for the reason above, I wouldn't replace existing functionality. Instead, maybe add a second bash array handled by it (in addition to binds, have symlinks or such).

Sounds good. That was also my proposal.

In past there was an AppArmor issue with symlinks rather than bind mount. It had been ported to symlinks then. Reference:

#1122

Thanks for the hint! I saw that issue before but did not really think about it in detail yet because I have not yet reinstalled AppArmor on Qubes OS. That is kind of a deal breaker then at least for my current use case because then also the AppArmor profile would need to be updated (which is already automated/needed for other reasons but if it can be avoided that would be better here).

Looking around in the Ansible docs I found that the template module has an option unsafe_writes which allows to update bind mounted files in place. So this sounds like the best option for my use case (using bind mounts and unsafe_writes).

I might look into symlinks again when the need arises. Thanks for the feedback!

[ypid]

But for the reason above, I wouldn't replace existing functionality. Instead, maybe add a second bash array handled by it (in addition to binds, have symlinks or such).

Sounds good. That was also my proposal.

In past there was an AppArmor issue with symlinks rather than bind mount. It had been ported to symlinks then. Reference:

#1122

Thanks for the hint! I saw that issue before but did not really think about it in detail yet because I have not yet reinstalled AppArmor on Qubes OS. That is kind of a deal breaker then at least for my current use case because then also the AppArmor profile would need to be updated (which is already automated/needed for other reasons but if it can be avoided that would be better here).

Looking around in the Ansible docs I found that the template module has an option unsafe_writes which allows to update bind mounted files in place. So this sounds like the best option for my use case (using bind mounts and unsafe_writes).

I might look into symlinks again when the need arises. Thanks for the feedback!

[Rudd-O]

Hey, by chance, are you using ansible-qubes? -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Rudd-O]

Hey, by chance, are you using ansible-qubes? -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[ypid]

Hey, by chance, are you using ansible-qubes?

Yes, I do. Thanks very much for you work! It allowed me to do my setup with Ansible as I did it for my bare metal Debian stable workstation before. See also: debops/debops-playbooks#333, https://github.com/Rudd-O/ansible-qubes/commits?author=ypid ;-)

[ypid]

Hey, by chance, are you using ansible-qubes?

Yes, I do. Thanks very much for you work! It allowed me to do my setup with Ansible as I did it for my bare metal Debian stable workstation before. See also: debops/debops-playbooks#333, https://github.com/Rudd-O/ansible-qubes/commits?author=ypid ;-)