Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign updon't really boot TemplateVM, only chroot into those #1306
Comments
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
marmarek
Oct 9, 2015
Member
I think it is a bad idea, for many reasons:
- Creating initial data is in many cases desirable. Remember that rootfs changes are not persistent in AppVMs, so when some service needs some more work at first startup, it will be done at each VM startup. Results in bad performance, long VM startup, waste of CPU power and high IO (especially important on laptops, with SSD). Some example of such service is
dnf-makecache- it creates cache of metadata from every installed and available package, takes a lot of time (minutes, or even tens minutes, 100% CPU). This one we have specifically disabled, but this isn't the only such service. And BTW we'll probably need to reenablednfin Fedora 22 template... - Starting TemplateVM is not always only about installing packages. Some users want to make system-wide changes. And some of them, especially when using GUI tools, require running (dbus) services which applies the changes. This is also about GUI updaters (#982), which we want to resurrect one day.
- Such approach would make the whole template update/installation actions really fragile. Need to track upstream changes, probably apply some hacks/workarounds for things not working in chroot.
- If you fear some initial random seeds creation, this approach doesn't help fully - because some packages creates them in post installation scripts. So it would be just half-solution, which isn't generic enough, but introduce a lot of potential problems (now and in the future).
- Qubes OS is not only for anonymity. In some cases (especially when one use a lot of VMs) higher priority is performance (VM startup time) over anonymity. This of course should be done in a way not compromising security (think - cryptography).
So, instead, I would suggest preparing some "blacklist" of packages known of creating unique initial data, which should really be unique for every VM. This list would be probably different for Whonix and non-Whonix templates. And then clean such data (it should be simple to create some service running at TemplateVM shutdown, which would remove/sanitize listed files). This isn't generic solution (neither is yours), but IMHO have much less potential for breaking random things (there are over 2k packages in template and probably only few of them require such treatment).
Another solution would be to move such (selected) data to /rw partition (either using symlinks or bind-mounts). Then each VM would have its own, persistent copy. But this would be probably much more service-specific and require more work.
|
I think it is a bad idea, for many reasons:
So, instead, I would suggest preparing some "blacklist" of packages known of creating unique initial data, which should really be unique for every VM. This list would be probably different for Whonix and non-Whonix templates. And then clean such data (it should be simple to create some service running at TemplateVM shutdown, which would remove/sanitize listed files). This isn't generic solution (neither is yours), but IMHO have much less potential for breaking random things (there are over 2k packages in template and probably only few of them require such treatment). Another solution would be to move such (selected) data to /rw partition (either using symlinks or bind-mounts). Then each VM would have its own, persistent copy. But this would be probably much more service-specific and require more work. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
adrelanos
Oct 10, 2015
Member
Marek Marczykowski-Górecki:
I think it is a bad idea, for many reasons:
Sure. The final decision is yours. Just trying to brainstorm here. I am
looking for the most generic, best maintainable, least code, you name
it, solution here. I try to specific some more. If my idea still doesn't
fly, just close this one.
- Creating initial data is in many cases desirable. Remember that
rootfs changes are not persistent in AppVMs, so when some service
needs some more work at first startup, it will be done at each VM
startup. Results in bad performance, long VM startup, waste of CPU
power and high IO (especially important on laptops, with SSD). Some
example of such service isdnf-makecache- it creates cache of
metadata from every installed and available package, takes a lot of
time (minutes, or even tens minutes, 100% CPU). This one we have
specifically
disabled,
but this isn't the only such service. And BTW we'll probably need to
reenablednfin Fedora 22 template...
Maybe there is a contradiction here or lack of understanding on my side.
On one hand, officially distributed image builds are created in chroot
and never booted [systemd services never run] before they get deployed.
That is considered fine by both of us.
If the user boots the template and installs packages, you prefer the
services (systemd...) to actually start. On the other hand, I suggest
these should be handled same as if in chroot.
But imagine a user just downloaded a fresh TemplateVM and created an
AppVM based on it. Then just starts the AppVM without ever starting the
TemplateVM beforehand. Doesn't that AppVM then miss everything you
mentioned?
- Starting TemplateVM is not
always only about installing packages. Some users want to make
system-wide changes. And some of them, especially when using GUI
tools, require running (dbus) services which applies the changes.
This is also about GUI updaters (#982), which we want to resurrect
one day.
For system-wide changes:
What I had in mind would run the terminal, file manager, etc. already
chrooted into the template image.
GUI tools, dbus in chroot:
I don't know if there would be a sane solution to just selectively and
dynamically start what the user requires. A quick search implies, that
GUI and dbus can be used within chroot.
What I am suggesting is basically just to start in the TemplateVM what
the user wishes, not all the system default. I.e. run in chroot,
apt-get, Apper with its dbus dependency, nautilus etc. As opposed to
starting the whole system default. That would for example exclude
openssh-server from starting and creating its private keys.
- Such approach would make the whole template
update/installation actions really fragile. Need to track upstream
changes, probably apply some hacks/workarounds for things not working
in chroot.
I don't think so. This is same as 1. If we consider package installation
during build in chroot fine, then package installation by the user in
chroot also should be fine? I've never came across a package that is not
installable inside chroot. I guess that would be considered as a release
critical bug by upstream.
- If you fear some initial random seeds creation, this
approach doesn't help fully - because some packages creates them in
post installation scripts. So it would be just half-solution, which
isn't generic enough, but introduce a lot of potential problems (now
and in the future).
I see.
- Qubes OS is not only for anonymity. In some
cases (especially when one use a lot of VMs) higher priority is
performance (VM startup time) over anonymity. This of course should
be done in a way not compromising security (think - cryptography).
Ok.
So, instead, I would suggest preparing some "blacklist" of packages
known of creating unique initial data, which should really be unique
for every VM.This list would be probably different for Whonix and
non-Whonix templates.
On first thought, I think if anything, Whonix would just extend that
list. Not use fewer. But even then, for now I cannot imagine a use case
where a service better runs in non-Whonix but not Whonix VMs or vice versa.
And then clean such data (it should be simple
to create some service running at TemplateVM shutdown, which would
remove/sanitize listed files).
Hm. Cleaning sounds difficult, package specific, maintenance burdensome,
test intensive, fragile. But maybe there is no other way.
If the unwanted files aren't created by post installation scripts, we
are better off adding a systemd drop-in file to prevent startup of the
service when run withing a TemplateVM.
This isn't generic solution (neither
is yours), but IMHO have much less potential for breaking random
things (there are over 2k packages in template and probably only few
of them require such treatment).
But there are X how many k packages available from installation from
upstream (Debian/Fedora) repositories. And the user is likely to install
problematic ones in the template. For example, openssh-server seems like
a very problematic case.
Another solution would be to move such (selected) data to /rw
partition (either using symlinks or bind-mounts). Then each VM would
have its own, persistent copy. But this would be probably much more
service-specific and require more work.
Yes.
Btw, there is such a bind-mount script. @nrgaway wrote it.
https://github.com/Whonix/qubes-whonix/blob/master/usr/lib/qubes-whonix/bind-directories
Not sure why it was committed to qubes-whonix rather than Qubes. I had
in mind to make it configurable and to to upstream it to Qubes.
|
Marek Marczykowski-Górecki:
Sure. The final decision is yours. Just trying to brainstorm here. I am
Maybe there is a contradiction here or lack of understanding on my side. If the user boots the template and installs packages, you prefer the But imagine a user just downloaded a fresh TemplateVM and created an
For system-wide changes: GUI tools, dbus in chroot: What I am suggesting is basically just to start in the TemplateVM what
I don't think so. This is same as 1. If we consider package installation
I see.
Ok.
On first thought, I think if anything, Whonix would just extend that
Hm. Cleaning sounds difficult, package specific, maintenance burdensome, If the unwanted files aren't created by post installation scripts, we
But there are X how many k packages available from installation from
Yes. Btw, there is such a bind-mount script. @nrgaway wrote it. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
marmarek
Oct 10, 2015
Member
- Creating initial data is in many cases desirable. Remember that
rootfs changes are not persistent in AppVMs, so when some service
needs some more work at first startup, it will be done at each VM
startup. Results in bad performance, long VM startup, waste of CPU
power and high IO (especially important on laptops, with SSD). Some
example of such service isdnf-makecache- it creates cache of
metadata from every installed and available package, takes a lot of
time (minutes, or even tens minutes, 100% CPU). This one we have
specifically
disabled,
but this isn't the only such service. And BTW we'll probably need to
reenablednfin Fedora 22 template...Maybe there is a contradiction here or lack of understanding on my side.
On one hand, officially distributed image builds are created in chroot
and never booted [systemd services never run] before they get deployed.
That is considered fine by both of us.
We have our template builder which can apply some post-installation
fixes. Granted, currently there is almost no such fixes. AFAIR only
setting locale, because by default it isn't set at all (defaults to
"C"), which prevent gnome-terminal from working.
If the user boots the template and installs packages, you prefer the
services (systemd...) to actually start. On the other hand, I suggest
these should be handled same as if in chroot.But imagine a user just downloaded a fresh TemplateVM and created an
AppVM based on it. Then just starts the AppVM without ever starting the
TemplateVM beforehand. Doesn't that AppVM then miss everything you
mentioned?
Not really - because the template is started during package installation
- mostly to retrieve applications list and icons, but also to create
initial data (to speed up later AppVMs boot).
And that's also the reason why some distributions provide some
post-installation configuration tools (firstboot/initial-setup in case
of Fedora) - this tool is specifically run in just installed system, not
during installation (using chroot).
- Starting TemplateVM is not
always only about installing packages. Some users want to make
system-wide changes. And some of them, especially when using GUI
tools, require running (dbus) services which applies the changes.
This is also about GUI updaters (#982), which we want to resurrect
one day.For system-wide changes:
What I had in mind would run the terminal, file manager, etc. already
chrooted into the template image.GUI tools, dbus in chroot:
I don't know if there would be a sane solution to just selectively and
dynamically start what the user requires. A quick search implies, that
GUI and dbus can be used within chroot.
Not sure if that's fully true. Calling dbus service requires that the
service is started (or already running). In case of system services
(like PackageKit, or systemd-timedated) this means that dbus system
daemon (first required service) forward a start request to systemd
(second running service), which starts the service (third service).
What I am suggesting is basically just to start in the TemplateVM what
the user wishes, not all the system default. I.e. run in chroot,
apt-get, Apper with its dbus dependency, nautilus etc. As opposed to
starting the whole system default. That would for example exclude
openssh-server from starting and creating its private keys.
Hmm, I don't think even that would be enough. During template build
process we specifically disable starting services:
https://github.com/QubesOS/qubes-builder-debian/blob/master/template_debian/distribution.sh#L91-L120
Without this code, during template build a bunch of services were
started, for example preventing unmounting the image.
This is mostly Debian-specific, because Debian policy is to start the
service (make it ready to use) just after installation. In Fedora
services are not started at installation (idea: user first may/should
configure it).
- Such approach would make the whole template
update/installation actions really fragile. Need to track upstream
changes, probably apply some hacks/workarounds for things not working
in chroot.I don't think so. This is same as 1. If we consider package installation
during build in chroot fine, then package installation by the user in
chroot also should be fine? I've never came across a package that is not
installable inside chroot. I guess that would be considered as a release
critical bug by upstream.
I have in mind mostly starting/running the process (all the surrounding
things, like proxy configuration, GUI tools (including terminal emulator
itself) etc. Not package installation itself, which indeed should just
work.
So, instead, I would suggest preparing some "blacklist" of packages
known of creating unique initial data, which should really be unique
for every VM.This list would be probably different for Whonix and
non-Whonix templates.On first thought, I think if anything, Whonix would just extend that
list.
Yes, that was my thought.
Not use fewer. But even then, for now I cannot imagine a use case
where a service better runs in non-Whonix but not Whonix VMs or vice versa.
I can't think of such example either for now. But I have in mind some
applications with first start wizards etc.
And then clean such data (it should be simple
to create some service running at TemplateVM shutdown, which would
remove/sanitize listed files).Hm. Cleaning sounds difficult, package specific, maintenance burdensome,
test intensive, fragile. But maybe there is no other way.If the unwanted files aren't created by post installation scripts, we
are better off adding a systemd drop-in file to prevent startup of the
service when run withing a TemplateVM.
Yes. And consider it either Qubes-wide, or only for Whonix. Case by
case.
This isn't generic solution (neither
is yours), but IMHO have much less potential for breaking random
things (there are over 2k packages in template and probably only few
of them require such treatment).But there are X how many k packages available from installation from
upstream (Debian/Fedora) repositories. And the user is likely to install
problematic ones in the template. For example, openssh-server seems like
a very problematic case.
Just double checked - currently chroot doesn't prevent host key
generation on openssh-server package install. See here:
https://gist.github.com/marmarek/b227dcff130862fa5b48
Another solution would be to move such (selected) data to /rw
partition (either using symlinks or bind-mounts). Then each VM would
have its own, persistent copy. But this would be probably much more
service-specific and require more work.Yes.
Btw, there is such a bind-mount script. @nrgaway wrote it.
https://github.com/Whonix/qubes-whonix/blob/master/usr/lib/qubes-whonix/bind-directories
Not sure why it was committed to qubes-whonix rather than Qubes. I had
in mind to make it configurable and to to upstream it to Qubes.
Good idea. Making it configurable seems to be quite easy, as in the
current shape it already have "configuration", just embedded instead of
separate file. Then Whonix could just insert configuration for it (using
drop-ins approach).
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
We have our template builder which can apply some post-installation
Not really - because the template is started during package installation
Not sure if that's fully true. Calling dbus service requires that the
Hmm, I don't think even that would be enough. During template build This is mostly Debian-specific, because Debian policy is to start the
I have in mind mostly starting/running the process (all the surrounding
Yes, that was my thought.
I can't think of such example either for now. But I have in mind some
Yes. And consider it either Qubes-wide, or only for Whonix. Case by
Just double checked - currently chroot doesn't prevent host key
Good idea. Making it configurable seems to be quite easy, as in the Best Regards, |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
adrelanos
Oct 10, 2015
Member
Good idea. Making it configurable seems to be quite easy, as in the current shape it already have "configuration", just embedded instead of separate file. Then Whonix could just insert configuration for it (using drop-ins approach).
Yes. (Added this to my plate - https://phabricator.whonix.org/T414 - but I not mind about anyone jumping in. A good, easy task for contributors to get started btw.)
Yes. (Added this to my plate - https://phabricator.whonix.org/T414 - but I not mind about anyone jumping in. A good, easy task for contributors to get started btw.) |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
adrelanos
Oct 10, 2015
Member
Closing this as wontfix then.
Thanks a lot for all the explanation and your consideration!
|
Closing this as wontfix then. Thanks a lot for all the explanation and your consideration! |
adrelanos commentedOct 9, 2015
I think booting a TemplateVM directly is bad. During boot, various daemons start [custom installed packages] and create their data folders, random seeds are created and so forth. All that date is then shared with AppVMs, so it could happen, that those all share the same random seeds.
So I am suggesting to not "really" boot into the TemplateVM. From the point of view of the user nothing should change. But when one boots a TemplateVM, what really should happen under the hood, is creating a temporary AppVM based on that Template. And then
chrooting into the TemplateVMs image. When one would update or install new packages inside the TemplateVM, these services would not start up, hence not make the image "dirty".