readonly root inside container #7923

Closed
alberts opened this Issue Sep 7, 2014 · 11 comments

Comments

Projects
None yet
4 participants
@alberts

alberts commented Sep 7, 2014

Fedora has supported the concept of a readonly root for a long time as way of making a system more stateless.

We have used this with boot-from-initramfs deployments to make sure that no service accidentally writes somewhere it shouldn't. I think this could also be useful within containers to make sure that services that run in the container are properly configured to either write data to tmpfs-backed temporary directories or to r/w volumes shared from outside with --volume and nowhere else.

The main Fedora script is here: https://git.fedorahosted.org/cgit/initscripts.git/tree/systemd/fedora-readonly

A writeup of how it works: https://warewolf.github.io/blog/2013/10/12/setting-up-a-read-only-rootfs-fedora-box/

In a Docker context, I think the main things that would carry over are flags for docker run:

  • roroot=true|false (mount root of container read-only)
  • rwtab=somefile (files and directories that should be bind-mounted so they are writable to a tmpfs)
  • statetab=somefile (files that should be writable to persistent storage
  • statemount=/some/dir (path to a directory passed through with --volume for state files)
  • rwoptions= (mount options for the tmpfs, size= is the main useful one)

Does this sound like something that's interesting to support in Docker? If so, I could start looking at putting together the start of a pull request.

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Sep 9, 2014

Contributor

I am +1 for this and it's already able to be supported in libconatiner.

Do you think we need to have the ability to specify tmpfs mounts first though via docker before we can do this? There will be some applications that need to write in /tmp or /run and it would be nice to have these mounted in tmpfs.

Contributor

crosbymichael commented Sep 9, 2014

I am +1 for this and it's already able to be supported in libconatiner.

Do you think we need to have the ability to specify tmpfs mounts first though via docker before we can do this? There will be some applications that need to write in /tmp or /run and it would be nice to have these mounted in tmpfs.

@alberts

This comment has been minimized.

Show comment
Hide comment
@alberts

alberts Sep 10, 2014

I think you can take care of passing the equivalent of Fedora's RW_MOUNT=/var/lib/stateless/writable and STATE_MOUNT=/var/lib/stateless/state through to the container using --volume flags.

There's still some configuration required. I think this is probably something that should go in the Dockerfile, but maybe people want to be able to change their minds at docker run time.

Taking a look at Fedora's rwtab today, you have 3 types of configuration:

dirs /var/log
empty /tmp
files /etc/resolv.conf

dirs means take all the directories in the base image and make them available for read/write, either on top of a tmpfs or on top of the RW_MOUNT directory.

empty means the same for the directory, but ignore any subdirectories that might exist in the base image.

files gives you a read/write file with the initial contents from the image.

So if you do this in the Dockerfile, each of these might become a RWTAB line (need a better name) you'd also have a RW_MOUNT specifying a directory, where if it isn't set, means you get a tmpfs.

And then maybe docker run takes a --rwmount flag. Maybe you also need to be able to do rwtab entries on the command line. Maybe --rw-dir --rw-empty --rw-files?

From my perspective, the main reason it's nice to have this in the Dockerfile is that it simplifies the arguments needed to get one of these containers running with the right rw and state mounts and rwtab entries, without changing anything in the Mesos API for running Docker containers.

alberts commented Sep 10, 2014

I think you can take care of passing the equivalent of Fedora's RW_MOUNT=/var/lib/stateless/writable and STATE_MOUNT=/var/lib/stateless/state through to the container using --volume flags.

There's still some configuration required. I think this is probably something that should go in the Dockerfile, but maybe people want to be able to change their minds at docker run time.

Taking a look at Fedora's rwtab today, you have 3 types of configuration:

dirs /var/log
empty /tmp
files /etc/resolv.conf

dirs means take all the directories in the base image and make them available for read/write, either on top of a tmpfs or on top of the RW_MOUNT directory.

empty means the same for the directory, but ignore any subdirectories that might exist in the base image.

files gives you a read/write file with the initial contents from the image.

So if you do this in the Dockerfile, each of these might become a RWTAB line (need a better name) you'd also have a RW_MOUNT specifying a directory, where if it isn't set, means you get a tmpfs.

And then maybe docker run takes a --rwmount flag. Maybe you also need to be able to do rwtab entries on the command line. Maybe --rw-dir --rw-empty --rw-files?

From my perspective, the main reason it's nice to have this in the Dockerfile is that it simplifies the arguments needed to get one of these containers running with the right rw and state mounts and rwtab entries, without changing anything in the Mesos API for running Docker containers.

@discordianfish

This comment has been minimized.

Show comment
Hide comment
@discordianfish

discordianfish Oct 22, 2014

Contributor

👍 Read-only rootfs would be very useful. It enforces usage of containers and guarantees you don't depend on the data stored in the container. It also make maintaining and cleaning up easier.

See also #2710

Contributor

discordianfish commented Oct 22, 2014

👍 Read-only rootfs would be very useful. It enforces usage of containers and guarantees you don't depend on the data stored in the container. It also make maintaining and cleaning up easier.

See also #2710

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Oct 23, 2014

Contributor

@discordianfish this is already possible in libcontainer. Can you think of a proposal for a flag, maybe:
docker run --readonly ubuntu bash ?

Contributor

crosbymichael commented Oct 23, 2014

@discordianfish this is already possible in libcontainer. Can you think of a proposal for a flag, maybe:
docker run --readonly ubuntu bash ?

@discordianfish

This comment has been minimized.

Show comment
Hide comment
@discordianfish

discordianfish Oct 24, 2014

Contributor

@crosbymichael --readonly sounds good!

Contributor

discordianfish commented Oct 24, 2014

@crosbymichael --readonly sounds good!

@scf37

This comment has been minimized.

Show comment
Hide comment
@scf37

scf37 Jan 10, 2015

Read-only containers are more secure as well.

scf37 commented Jan 10, 2015

Read-only containers are more secure as well.

@discordianfish

This comment has been minimized.

Show comment
Hide comment
@discordianfish

discordianfish Jan 11, 2015

Contributor

@crosbymichael Is this something we want and just nobody has implemented it? If so, I can do that.

Contributor

discordianfish commented Jan 11, 2015

@crosbymichael Is this something we want and just nobody has implemented it? If so, I can do that.

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Jan 12, 2015

Contributor

@discordianfish it's implemented ( in libcontainer ), we just need some agreement on the flag.

Let me discuss with my fellow maintainers and see if we can get this sorted out and in for 1.5

Contributor

crosbymichael commented Jan 12, 2015

@discordianfish it's implemented ( in libcontainer ), we just need some agreement on the flag.

Let me discuss with my fellow maintainers and see if we can get this sorted out and in for 1.5

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Jan 12, 2015

Contributor

I think this will probably depend on tmpfs mounts also because /tmp will probably have to go into tmpfs

Contributor

crosbymichael commented Jan 12, 2015

I think this will probably depend on tmpfs mounts also because /tmp will probably have to go into tmpfs

@crosbymichael crosbymichael added this to the 1.5.0 milestone Jan 12, 2015

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Jan 13, 2015

Contributor

@discordianfish go ahead and implement --readonly if you have time. For it to be in docker 1.5 it needs to be done this week, if you don't have time I can implement it.

Contributor

crosbymichael commented Jan 13, 2015

@discordianfish go ahead and implement --readonly if you have time. For it to be in docker 1.5 it needs to be done this week, if you don't have time I can implement it.

@discordianfish

This comment has been minimized.

Show comment
Hide comment
@discordianfish

discordianfish Jan 14, 2015

Contributor

@crosbymichael I would love to implement it, but have no time this week. So if you have time, go for it.

Contributor

discordianfish commented Jan 14, 2015

@crosbymichael I would love to implement it, but have no time this week. So if you have time, go for it.

crosbymichael added a commit to crosbymichael/docker that referenced this issue Jan 14, 2015

Add --readonly for read only container rootfs
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly.  This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted.  This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.

Closes #7923
Closes #8752

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

crosbymichael added a commit to crosbymichael/docker that referenced this issue Jan 14, 2015

Add --readonly for read only container rootfs
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly.  This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted.  This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.

Closes #7923
Closes #8752

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

crosbymichael added a commit to crosbymichael/docker that referenced this issue Jan 14, 2015

Add --readonly for read only container rootfs
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly.  This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted.  This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.

Closes #7923
Closes #8752

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

crosbymichael added a commit to crosbymichael/docker that referenced this issue Jan 14, 2015

Add --readonly for read only container rootfs
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly.  This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted.  This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.

Closes #7923
Closes #8752

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

crosbymichael added a commit to crosbymichael/docker that referenced this issue Jan 14, 2015

Add --readonly for read only container rootfs
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly.  This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted.  This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.

Closes #7923
Closes #8752

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

crosbymichael added a commit to crosbymichael/docker that referenced this issue Jan 14, 2015

Add --readonly for read only container rootfs
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly.  This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted.  This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.

Closes #7923
Closes #8752

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

crosbymichael added a commit to crosbymichael/docker that referenced this issue Jan 14, 2015

Add --readonly for read only container rootfs
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly.  This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted.  This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.

Closes #7923
Closes #8752

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

@LK4D4 LK4D4 closed this in #10093 Jan 14, 2015

@wking wking referenced this issue in opencontainers/runtime-spec Jun 25, 2015

Closed

Clarify readonlyRootfs #23

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment