rkt: `run` as unprivileged user #1585

Open
jonboulle opened this Issue Oct 9, 2015 · 13 comments

Comments

Projects
None yet
5 participants
@jonboulle
Contributor

jonboulle commented Oct 9, 2015

Recent changes to rkt have allowed us to perform a large subset of rkt commands (image list, fetch, etc) as an unprivileged user. It would be nice if it was possible to do ALL rkt operations this way: the elephant in the room is run, which historically requires root privileges for several operations (chroot, setting up network namespaces).

For the default rkt stage1, this would likely be predicated on systemd-nspawn gaining this ability. Relevant systemd thread: http://lists.freedesktop.org/archives/systemd-devel/2015-February/028024.html

Another option is to explore using unc to create containers: #1318

@jellonek

This comment has been minimized.

Show comment
Hide comment
@jellonek

jellonek Oct 12, 2015

Contributor

Network configuration could be done by plugins with set uid bit set on them in most cases, or with set uid bit set on stage1/init in lkvm case.
Chroot also can be done with similar small C utility with this same bit set.

Contributor

jellonek commented Oct 12, 2015

Network configuration could be done by plugins with set uid bit set on them in most cases, or with set uid bit set on stage1/init in lkvm case.
Chroot also can be done with similar small C utility with this same bit set.

@vbatts

This comment has been minimized.

Show comment
Hide comment
@vbatts

vbatts Feb 12, 2016

Contributor

another option for a runtime could be https://git.gnome.org/browse/linux-user-chroot

Contributor

vbatts commented Feb 12, 2016

another option for a runtime could be https://git.gnome.org/browse/linux-user-chroot

@alban

This comment has been minimized.

Show comment
Hide comment
@alban

alban Feb 16, 2016

Member

rkt run also needs root to untar an ACI with proper ownership permissions.

Member

alban commented Feb 16, 2016

rkt run also needs root to untar an ACI with proper ownership permissions.

@jellonek

This comment has been minimized.

Show comment
Hide comment
@jellonek

jellonek Feb 16, 2016

Contributor

So there are 4 things which require upper rights:

  • aci extraction
  • call to CNI (in both cases, run/gc time)
  • mounts/binds
  • pod execution process

It's the good moment to plan how to refactor and extract these functionalities into external commands, which could be started using suid bit, and which will throw upper rights right after primary job is done.

Contributor

jellonek commented Feb 16, 2016

So there are 4 things which require upper rights:

  • aci extraction
  • call to CNI (in both cases, run/gc time)
  • mounts/binds
  • pod execution process

It's the good moment to plan how to refactor and extract these functionalities into external commands, which could be started using suid bit, and which will throw upper rights right after primary job is done.

@muayyad-alsadi

This comment has been minimized.

Show comment
Hide comment
@muayyad-alsadi

muayyad-alsadi Feb 16, 2016

Will apache needs root privilege to open port 80 but we end with it running
as apache user.

Can't we have something like "--become=user2" to the user running the
container just before entering the name space?

So I run rkt as root and pass it --become.

Regarding extracting aci. Maybe I can pass custom path.
On Feb 16, 2016 4:51 PM, "Piotr Skamruk" notifications@github.com wrote:

So there are 4 things which require upper rights:

  • aci extraction
  • call to CNI (in both cases, run/gc time)
  • mounts/binds
  • pod execution process

It's the good moment to plan how to refactor and extract these
functionalities into external commands, which could be started using suid
bit, and which will throw upper rights right after primary job is done.


Reply to this email directly or view it on GitHub
#1585 (comment).

Will apache needs root privilege to open port 80 but we end with it running
as apache user.

Can't we have something like "--become=user2" to the user running the
container just before entering the name space?

So I run rkt as root and pass it --become.

Regarding extracting aci. Maybe I can pass custom path.
On Feb 16, 2016 4:51 PM, "Piotr Skamruk" notifications@github.com wrote:

So there are 4 things which require upper rights:

  • aci extraction
  • call to CNI (in both cases, run/gc time)
  • mounts/binds
  • pod execution process

It's the good moment to plan how to refactor and extract these
functionalities into external commands, which could be started using suid
bit, and which will throw upper rights right after primary job is done.


Reply to this email directly or view it on GitHub
#1585 (comment).

@alban

This comment has been minimized.

Show comment
Hide comment
@alban

alban Feb 16, 2016

Member

It's the good moment to plan how to refactor and extract these functionalities into external commands, which could be started using suid bit, and which will throw upper rights right after primary job is done

Replacing root by external setuid binaries seems quite dangerous to me. Each of those external setuid binaries could easily be instructed to do bad things. They would need to have enough context to decide whether it's safe.

As comparison, the setuid /usr/bin/mount binary knows it is allowed to mount something as user if "user" is specified in fstab. For each external setuid component, how would they decide?

I like that rkt trust, rkt fetch, rkt image list, etc. can do the downloads and image verification without root. But I'm not sure what is to gain from further separation.

If a setuid aci-extraction binary can extract any ACI in /var/lib/rkt/cas/tree (instead of the current model with rkt running as root), it will need to block any alternative path for --dir=/var/lib/rkt or ensure that the directory is not readable by normal users. Otherwise, any users could extract an ACI containing /dev/sda with rwx rights and then write on it. It will need to check it is not running in a chroot as well. The other setuid components will bring additional security considerations as well.

Member

alban commented Feb 16, 2016

It's the good moment to plan how to refactor and extract these functionalities into external commands, which could be started using suid bit, and which will throw upper rights right after primary job is done

Replacing root by external setuid binaries seems quite dangerous to me. Each of those external setuid binaries could easily be instructed to do bad things. They would need to have enough context to decide whether it's safe.

As comparison, the setuid /usr/bin/mount binary knows it is allowed to mount something as user if "user" is specified in fstab. For each external setuid component, how would they decide?

I like that rkt trust, rkt fetch, rkt image list, etc. can do the downloads and image verification without root. But I'm not sure what is to gain from further separation.

If a setuid aci-extraction binary can extract any ACI in /var/lib/rkt/cas/tree (instead of the current model with rkt running as root), it will need to block any alternative path for --dir=/var/lib/rkt or ensure that the directory is not readable by normal users. Otherwise, any users could extract an ACI containing /dev/sda with rwx rights and then write on it. It will need to check it is not running in a chroot as well. The other setuid components will bring additional security considerations as well.

@muayyad-alsadi

This comment has been minimized.

Show comment
Hide comment
@muayyad-alsadi

muayyad-alsadi Feb 16, 2016

what I'm thinking about is not a setuid nor daemon.

I run rkt as root and ask it to become another specific user when it can
just before running the container. One use case is to run untrusted
container with a tenant specific user so the scope of damage is for that
tenant
On Feb 16, 2016 6:28 PM, "Alban Crequy" notifications@github.com wrote:

It's the good moment to plan how to refactor and extract these
functionalities into external commands, which could be started using suid
bit, and which will throw upper rights right after primary job is done

Replacing root by external setuid binaries seems quite dangerous to me.
Each of those external setuid binaries could easily be instructed to do bad
things. They would need to have enough context to decide whether it's safe.

As comparison, the setuid /usr/bin/mount binary knows it is allowed to
mount something as user if "user" is specified in fstab. For each external
setuid component, how would they decide?

I like that rkt trust, rkt fetch, rkt image list, etc. can do the
downloads and image verification without root. But I'm not sure what is to
gain from further separation.

If a setuid aci-extraction binary can extract any ACI in
/var/lib/rkt/cas/tree (instead of the current model with rkt running as
root), it will need to block any alternative path for --dir=/var/lib/rkt
or ensure that the directory is not readable by normal users. Otherwise,
any users could extract an ACI containing /dev/sda with rwx rights and
then write on it. It will need to check it is not running in a chroot as
well. The other setuid components will bring additional security
considerations as well.


Reply to this email directly or view it on GitHub
#1585 (comment).

what I'm thinking about is not a setuid nor daemon.

I run rkt as root and ask it to become another specific user when it can
just before running the container. One use case is to run untrusted
container with a tenant specific user so the scope of damage is for that
tenant
On Feb 16, 2016 6:28 PM, "Alban Crequy" notifications@github.com wrote:

It's the good moment to plan how to refactor and extract these
functionalities into external commands, which could be started using suid
bit, and which will throw upper rights right after primary job is done

Replacing root by external setuid binaries seems quite dangerous to me.
Each of those external setuid binaries could easily be instructed to do bad
things. They would need to have enough context to decide whether it's safe.

As comparison, the setuid /usr/bin/mount binary knows it is allowed to
mount something as user if "user" is specified in fstab. For each external
setuid component, how would they decide?

I like that rkt trust, rkt fetch, rkt image list, etc. can do the
downloads and image verification without root. But I'm not sure what is to
gain from further separation.

If a setuid aci-extraction binary can extract any ACI in
/var/lib/rkt/cas/tree (instead of the current model with rkt running as
root), it will need to block any alternative path for --dir=/var/lib/rkt
or ensure that the directory is not readable by normal users. Otherwise,
any users could extract an ACI containing /dev/sda with rwx rights and
then write on it. It will need to check it is not running in a chroot as
well. The other setuid components will bring additional security
considerations as well.


Reply to this email directly or view it on GitHub
#1585 (comment).

@alban

This comment has been minimized.

Show comment
Hide comment
@alban

alban Feb 16, 2016

Member

@muayyad-alsadi rkt can already run apps as non-root by specifying a non-zero uid/gid in the app manifest. There is more ongoing work on this via #1919 and #2159. But true, apache cannot normally listen on port 80 if it runs as non-root (unless it is socket activated because in that case the socket is bound and listening before apache starts).

Member

alban commented Feb 16, 2016

@muayyad-alsadi rkt can already run apps as non-root by specifying a non-zero uid/gid in the app manifest. There is more ongoing work on this via #1919 and #2159. But true, apache cannot normally listen on port 80 if it runs as non-root (unless it is socket activated because in that case the socket is bound and listening before apache starts).

@muayyad-alsadi

This comment has been minimized.

Show comment
Hide comment
@muayyad-alsadi

muayyad-alsadi Feb 16, 2016

How apache work? Open port 80 as root then drop privilege.

a non-zero uid/gid in the app manifest.

Use case: I want a container running as a user called client1 the container
is running apache server as container's root user on port 80 and apache
drops privilege to be running as apache.

This is different than running apache server on 8080 as non root by app
manifest

On Tue, Feb 16, 2016, 6:52 PM Alban Crequy notifications@github.com wrote:

@muayyad-alsadi https://github.com/muayyad-alsadi rkt can already run
apps as non-root by specifying a non-zero uid/gid in the app manifest.
There is more ongoing work on this via #1919
#1919 and #2159
#2159. But true, apache cannot
normally listen on port 80 if it runs as non-root (unless it is socket
activated
https://github.com/coreos/rkt/blob/master/Documentation/using-rkt-with-systemd.md#socket-activated-service
because in that case the socket i s bound and listening before apache
starts).


Reply to this email directly or view it on GitHub
#1585 (comment).

How apache work? Open port 80 as root then drop privilege.

a non-zero uid/gid in the app manifest.

Use case: I want a container running as a user called client1 the container
is running apache server as container's root user on port 80 and apache
drops privilege to be running as apache.

This is different than running apache server on 8080 as non root by app
manifest

On Tue, Feb 16, 2016, 6:52 PM Alban Crequy notifications@github.com wrote:

@muayyad-alsadi https://github.com/muayyad-alsadi rkt can already run
apps as non-root by specifying a non-zero uid/gid in the app manifest.
There is more ongoing work on this via #1919
#1919 and #2159
#2159. But true, apache cannot
normally listen on port 80 if it runs as non-root (unless it is socket
activated
https://github.com/coreos/rkt/blob/master/Documentation/using-rkt-with-systemd.md#socket-activated-service
because in that case the socket i s bound and listening before apache
starts).


Reply to this email directly or view it on GitHub
#1585 (comment).

@jellonek

This comment has been minimized.

Show comment
Hide comment
@jellonek

jellonek Feb 17, 2016

Contributor

@alban: to be precise, there is one another option, setcap as in http://stackoverflow.com/a/414258

Contributor

jellonek commented Feb 17, 2016

@alban: to be precise, there is one another option, setcap as in http://stackoverflow.com/a/414258

@muayyad-alsadi

This comment has been minimized.

Show comment
Hide comment
@muayyad-alsadi

muayyad-alsadi Feb 19, 2016

Seeds for thought

http://lk4d4.darth.io/posts/unpriv1/
https://github.com/LK4D4/unc
On Feb 17, 2016 3:09 PM, "Piotr Skamruk" notifications@github.com wrote:

@alban https://github.com/alban: to be precise, there is one another
option, setcap as in http://stackoverflow.com/a/414258


Reply to this email directly or view it on GitHub
#1585 (comment).

Seeds for thought

http://lk4d4.darth.io/posts/unpriv1/
https://github.com/LK4D4/unc
On Feb 17, 2016 3:09 PM, "Piotr Skamruk" notifications@github.com wrote:

@alban https://github.com/alban: to be precise, there is one another
option, setcap as in http://stackoverflow.com/a/414258


Reply to this email directly or view it on GitHub
#1585 (comment).

@jellonek

This comment has been minimized.

Show comment
Hide comment
@jellonek

jellonek Feb 22, 2016

Contributor

We have this already as idea in #1318

Contributor

jellonek commented Feb 22, 2016

We have this already as idea in #1318

@muayyad-alsadi

This comment has been minimized.

Show comment
Hide comment
@muayyad-alsadi

muayyad-alsadi Feb 22, 2016

We have this already as idea in #1318

Not just as a limited stage1, but doing things that needs privilege like
setting network before changing user and calling unc.

On Mon, Feb 22, 2016, 11:07 AM Piotr Skamruk notifications@github.com
wrote:

We have this already as idea in #1318
#1318


Reply to this email directly or view it on GitHub
#1585 (comment).

We have this already as idea in #1318

Not just as a limited stage1, but doing things that needs privilege like
setting network before changing user and calling unc.

On Mon, Feb 22, 2016, 11:07 AM Piotr Skamruk notifications@github.com
wrote:

We have this already as idea in #1318
#1318


Reply to this email directly or view it on GitHub
#1585 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment