Chrome - Failed to move to new namespace. #65

Closed
xcellardoor opened this Issue Sep 23, 2015 · 77 comments

Projects

None yet
@xcellardoor

I keep getting this error when trying to run the Chrome image. If I run with the --no-sandbox argument for Chrome, it then complains about running as the root user and dies immediately. What's the solution?

Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted

@jessfraz
Owner

The arguments to run in my dotfiles work for me

On Wednesday, September 23, 2015, xcellardoor notifications@github.com
wrote:

I keep getting this error when trying to run the Chrome image. If I run
with the --no-sandbox argument for Chrome, it then complains about running
as the root user and dies immediately. What's the solution?

Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted


Reply to this email directly or view it on GitHub
#65.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@jessfraz
Owner

You should not be using --no-sandbox

On Wednesday, September 23, 2015, xcellardoor notifications@github.com
wrote:

I keep getting this error when trying to run the Chrome image. If I run
with the --no-sandbox argument for Chrome, it then complains about running
as the root user and dies immediately. What's the solution?

Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted


Reply to this email directly or view it on GitHub
#65.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@xcellardoor

Nope sorry still not working. I did a 'make' on your dotfiles and that all seems to have installed okay. Tried re-launching your Chrome and it failed with the same PID issue. Here is the end of installing the dotfiles and then starting the your chrome.

sudo systemctl daemon-reload
╭─cellardoor@glow  ~/github/docker/dotfiles  ‹master› 
╰─$ sudo systemctl daemon-reload 
╭─cellardoor@glow  ~/github/docker/dotfiles  ‹master› 
╰─$ docker run -it --net host --cpuset-cpus 0 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY --device /dev/snd jfrazelle/chrome                                                                      1 ↵
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
^C
@jessfraz
Owner

hmmm gotta be something weird w your setup :/

On Wed, Sep 23, 2015 at 11:15 AM, xcellardoor notifications@github.com
wrote:

Nope sorry still not working. I did a 'make' on your dotfiles and that all
seems to have installed okay. Tried re-launching your Chrome and it failed
with the same PID issue. Here is the end of installing the dotfiles and
then starting the your chrome.

sudo systemctl daemon-reload
╭─cellardoor@glow ~/github/docker/dotfiles ‹master›
╰─$ sudo systemctl daemon-reload
╭─cellardoor@glow ~/github/docker/dotfiles ‹master›
╰─$ docker run -it --net host --cpuset-cpus 0 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY --device /dev/snd jfrazelle/chrome 1 ↵
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
^C


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@xcellardoor

It's a relatively standard Arch Linux machine, nothing particularly weird about it unfortunately :(

You're not running as a special user are you? Online I've read it's possible to use --no-sandbox to remove the PID error but of course you're forgoing basically everything useful about sandboxing when you do that :| It then says I'm trying to run it as Root too, if using sandboxing. Are you invoking any commands to have it run as a regular user? Is your Docker daemon itself configured differently to stock perhaps?

Thanks.

@jessfraz
Owner

i use everything in my dotfiles its a standard docker run

On Wed, Sep 23, 2015 at 11:26 AM, xcellardoor notifications@github.com
wrote:

It's a relatively standard Arch Linux machine, nothing particularly weird
about it unfortunately :(

You're not running as a special user are you? Online I've read it's
possible to use --no-sandbox to remove the PID error but of course you're
forgoing basically everything useful about sandboxing when you do that :|
It then says I'm trying to run it as Root too, if using sandboxing. Are you
invoking any commands to have it run as a regular user? Is your Docker
daemon itself configured differently to stock perhaps?

Thanks.


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@xcellardoor

I tried this here for Chrome and it's working (with --no-sandbox, which is evil I know). The extra steps he does is to put a user in the container and then run as that user, it's got to be something to do with that. Just FYI really in case it's ever useful. http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/

@hurricanehrndz

Can confirm that UID and GID is necessary. Also /dev/dri mapping is also essential otherwise you will have Chrome crashing every couple of tabs.

@xcellardoor

Would you mind sharing how you passed the UID and GID through? What's the exact command you're using to startup this container? Thanks.

@hurricanehrndz

@xcellardoor

Yeah, I don't mind at all. I actually forked someone's else project and introduce a bunch of fix and features I thought he was missing. You can find the project here:
https://github.com/hurricanehrndz/docker-browser-box

Let me know if you encounter any issues, actually just open one up. Also the readme should be pretty self explanatory. I have not updated the README to point to my docker repo yet, but in case your wondering my username is hurricane on the hub. Feel free to fork though and make pull request.

@xcellardoor

@hurricanehrndz Sorry to abuse Jess' thread, but I can't figure out how to raise an issue with docker-browser-box on your page. Have you turned anything off? Getting some errors now. Could you let me know how to message/raise an issue so I can help resolve the problem. Thanks.

@hurricanehrndz

@xcellardoor issues are now open. My apologies go @jfrazelle and many thanks because without your hard work none of this would be possible!

@vikstrous

I'm getting this error on arch linux too. I tried to strace it and here's my understanding of the problem:

It thinks that it's able to create network namespaces and PID namespaces, but it fails when it tries to fork with clone() it fails with the following error:

[pid    20] clone(child_stack=0, flags=CLONE_NEWPID|CLONE_NEWNET|SIGCHLD) = -1 EPERM (Operation not permitted)

I'll try to reproduce the error by writing a small C program that does just the clone part.

@jessfraz
Owner
jessfraz commented Oct 5, 2015

its gotta be distro specific, i use this image everyday :/ even right now

@xcellardoor

The question is what does Arch do that Debian (which Jess uses) doesn't. I know Debian tends to value stability and older versions of packages... perhaps there is a complication with a package which cutting-edge Arch uses which Debian etc may soon hit too. I look forward to seeing what @vikstrous finds but if I can test anything for anyone, please ask and I'll be happy to.

@vikstrous

After reading the man page for clone I found out that chrome needs CAP_SYS_ADMIN to use the CLONE_NEWNET flag. I added --cap-add SYS_ADMIN and I got past the Operation not permitted error. I still get this though:

[1:1:1006/032221:ERROR:nacl_fork_delegate_linux.cc(314)] Bad NaCl helper startup ack (0 bytes)
@jessfraz
Owner
jessfraz commented Oct 6, 2015

All of this seems very wrong I do not add that cap what user are you
running as in the container, do you have apparmor or selinux installed?

On Monday, October 5, 2015, Viktor Stanchev notifications@github.com
wrote:

After reading the man page for clone I found out that chrome needs
CAP_SYS_ADMIN to use the CLONE_NEWNET flag. I added --cap-add SYS_ADMIN
and I got past the Operation not permitted error. I still get this though:

[1:1:1006/032221:ERROR:nacl_fork_delegate_linux.cc(314)] Bad NaCl helper startup ack (0 bytes)


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@jessfraz
Owner
jessfraz commented Oct 6, 2015

Mine creates the chrome sandbox just fine I just straced it

On Monday, October 5, 2015, Jessica Frazelle me@jessfraz.com wrote:

All of this seems very wrong I do not add that cap what user are you
running as in the container, do you have apparmor or selinux installed?

On Monday, October 5, 2015, Viktor Stanchev <notifications@github.com
javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

After reading the man page for clone I found out that chrome needs
CAP_SYS_ADMIN to use the CLONE_NEWNET flag. I added --cap-add SYS_ADMIN
and I got past the Operation not permitted error. I still get this
though:

[1:1:1006/032221:ERROR:nacl_fork_delegate_linux.cc(314)] Bad NaCl helper startup ack (0 bytes)


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu
http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@vikstrous

I don't have either one. Arch doesn't have them by default.

@vikstrous

Here's the c file to test with:

https://gist.github.com/vikstrous/151b4c74fc0ab4c10d85

Does this work on your system without --cap-add SYS_ADMIN ?

@xcellardoor

Thank you very much @vikstrous for writing the test. Here are the results:

cellardoor at glow in [~/Temp]
12:01:56 › gcc test.c

cellardoor at glow in [~/Temp]
12:02:02 › ./a.out
Cloning...
Cloning failed with errno 1: Operation not permitted

cellardoor at glow in [~/Temp]
12:02:05 › sudo ./a.out
Cloning...
Parent running.
Child running.
Test successful.

There is a problem with my user being unable to clone the process, but running with sudo as a whim to bypass whatever protection is in place worked.

@jessfraz
Owner
jessfraz commented Oct 6, 2015

Running 'unshare' will also clone just fyi

On Tuesday, October 6, 2015, xcellardoor notifications@github.com wrote:

Thank you very much @vikstrous https://github.com/vikstrous for writing
the test. Here are the results:

cellardoor at glow in [~/Temp]

12:01:56 › gcc test.c

cellardoor at glow in [~/Temp]

12:02:02 › ./a.out
Cloning...
Cloning failed with errno 1: Operation not permitted

cellardoor at glow in [~/Temp]

12:02:05 › sudo ./a.out
Cloning...
Parent running.
Child running.
Test successful.

There is a problem with my user being unable to clone the process, but
running with sudo as a whim to bypass whatever protection is in place
worked.


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@jessfraz
Owner
jessfraz commented Oct 6, 2015

this is the exact command i run, and i am posting this comment from the chrome in a container :p

    docker run -d \
        --memory 3gb \
        --net host \
        -v /etc/localtime:/etc/localtime:ro \
        -v /tmp/.X11-unix:/tmp/.X11-unix \
        -e DISPLAY=unix$DISPLAY \
        -v $HOME/Downloads:/root/Downloads \
        -v $HOME/Pictures:/root/Pictures \
        -v $HOME/Torrents:/root/Torrents \
        -v $HOME/.chrome:/data \
        -v /dev/shm:/dev/shm \
        -v /etc/hosts:/etc/hosts \
        --device /dev/snd \
        --device /dev/dri \
        --device /dev/video0 \
        --group-add audio \
        --group-add video \
        --name chrome \
        jess/chrome --user-data-dir=/data --force-device-scale-factor=1 \
        --proxy-server="$proxy" --host-resolver-rules="$map" "$args"

@vikstrous are u in the office tommorrow we can debug at lunch or something because it seems weird to me

@jessfraz
Owner
jessfraz commented Oct 6, 2015

also what kernel are you on?

@jessfraz
Owner
jessfraz commented Oct 6, 2015

and what docker version?

@vikstrous

Yes, --net host is off. This was on my personal laptop and I don't have it here, but I think the kernel is around 4.2.2 and the docker version is 1.8.2.

On October 6, 2015 10:40:57 AM PDT, Jess Frazelle notifications@github.com wrote:

and what docker version?


Reply to this email directly or view it on GitHub:
#65 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

@vikstrous

I tried Ubuntu in a VM. Here's what I found:

  • I confirmed that Chrome on Ubuntu starts without any special capabilities and without turning off the sandbox.
  • I confirmed that my test binary fails even though Chrome starts with the same config.
  • I don't think --net host changes anything WRT the sandbox
  • I have no idea how X11 does authentication on Ubuntu because clearly it doesn't use .Xauthority
  • I found that I couldn't strace inside the container without --privileged which is kind of weird. This is a bit off topic and it's a kernel issue, but it should be allowed IMO.
  • Arch Linux's kernel is compiled without user namespaces, but Ubuntu has them on. The issue seems to be that CLONE_NEWPID and CLONE_NEWNET are allowed only with CLONE_NEWUSER.

Chrome under ubuntu does a test for them from what I can see:

clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = 21

On arch this test fails:

clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = -1 EINVAL (Invalid argument)

Then on Ubuntu chrome does:

clone(child_stack=0x7fff9c88d7c0, flags=CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET|SIGCHLD) = 22

And on arch it tries:

clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f9d90f60d10) = 20

Now I know why this behaviour happens. The next question is if there is anything we can do about it.

I see two options that are currently possible for Arch users:

  • Using a custom kernel on Arch
  • Turning off the sandboxing on Arch

I see a few long term options:

  • Figure out how to make Chrome/Chromium fail more elegantly in this scenario and contribute a patch.
  • Get user namespaces enabled in the Arch kernel - https://bugs.archlinux.org/task/36969 - might take a year until things are more settled down and they are comfortable with the security of the feature.
  • I don't know if this is possible, and it's a kernel change, but maybe allow network/pid namespacing without CAP_SYS_ADMIN.
@vikstrous

Should chrome be allowed to even do clone with CLONE_NEWPID | CLONE_NEWNET | CLONE_NEWUSER? I'm not that familiar with how user namespaces are implemented, so I need to read more before I can figure out if this is normal.

Edit, I think this is indeed normal. This article cleared things up for me: http://lwn.net/Articles/528078/

@jessfraz
Owner
jessfraz commented Oct 7, 2015

Yeah all that makes sense and you definitely need CLONE_NEWUSER lol I
didn't realize that was missing from the kernel

So mine is using chromiums new namespace sandbox and yours is using the old setuid sandbox which is not as great :p

On Wednesday, October 7, 2015, Viktor Stanchev notifications@github.com
wrote:

Should chrome be allowed to even do clone with CLONE_NEWPID |
CLONE_NEWNET | CLONE_NEWUSER? I'm not that familiar with how user
namespaces are implemented, so I need to read more before I can figure out
if this is normal.


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@xcellardoor

Guessing you guys are on a roll in pinpointing the problem :)

@jfrazelle Tested your command this morning, turned off the -d flag so I could see what is put out on the CLI. Same problem as before so it definitely looks to be a sandboxing/namespaces issue on Arch. I could take a look into popping a custom kernel on here, it takes a few minutes so I don't mind.

Can either of you recommend any community built kernels that you know wouldn't implement these protections? I'm taking a look at Zen Kernel at the moment, for which Arch has a repo package. So far my Google-fu hasn't got a definitive answer. To make sure I understand, I'm after user-namespaces ON, but a greater priority is sandboxing OFF?

If it helps, my versions are:
Kernel - 4.2.2-1-ARCH
Docker - 1.8.2

cellardoor at glow in [~]   
10:05:08 ›     docker run \   
        --memory 3gb \
        --net host \
        -v /etc/localtime:/etc/localtime:ro \
        -v /tmp/.X11-unix:/tmp/.X11-unix \
        -e DISPLAY=unix$DISPLAY \
        -v $HOME/Downloads:/root/Downloads \
        -v $HOME/Pictures:/root/Pictures \
        -v $HOME/Torrents:/root/Torrents \
        -v $HOME/.chrome:/data \
        -v /dev/shm:/dev/shm \
        -v /etc/hosts:/etc/hosts \
        --device /dev/snd \
        --device /dev/dri \
        --device /dev/video0 \
        --group-add audio \
        --group-add video \
        --name chrome \
        jess/chrome --user-data-dir=/data --force-device-scale-factor=1 \
        --proxy-server="$proxy" --host-resolver-rules="$map" "$args"
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
@jessfraz
Owner
jessfraz commented Oct 7, 2015

Using a custom kernel is your best bet here, also I use custom kernels :p

Turning off sandboxing probably isn't the best idea

I tried submitting a patch to chromium once and nothing happened granted
it wasn't for this but still... Idk

You could submit a patch to arch kernel of make your own arch package with
a different kernel config

No, they would never allow cloning new net namespaces or new pid namespaces
without CAP_SYSADMIN

On Tuesday, October 6, 2015, Viktor Stanchev notifications@github.com
wrote:

I tried Ubuntu in a VM. Here's what I found:

  • I confirmed that Chrome on Ubuntu starts without any special
    capabilities and without turning off the sandbox.
  • I confirmed that my test binary fails even though Chrome starts with
    the same config.
  • I don't think --net host changes anything WRT the sandbox
  • I have no idea how X11 does authentication on Ubuntu because clearly
    it doesn't use .Xauthority
  • I found that I couldn't strace inside the container without
    --privileged which is kind of weird. This is a bit off topic and it's
    a kernel issue, but it should be allowed IMO.
  • Arch Linux's kernel is compiled without user namespaces, but Ubuntu
    has them on. The issue seems to be that CLONE_NEWPID and CLONE_NEWNET
    are allowed only with CLONE_NEWUSER.

Chrome under ubuntu does a test for them from what I can see:

clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = 21

On arch this test fails:

clone(child_stack=0, flags=CLONE_NEWUSER|SIGCHLD) = -1 EINVAL (Invalid argument)

Then on Ubuntu chrome does:

clone(child_stack=0x7fff9c88d7c0, flags=CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET|SIGCHLD) = 22

And on arch it tries:

clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f9d90f60d10) = 20

Now I know why this behaviour happens. The next question is if there is
anything we can do about it.

I see two options that are currently possible for Arch users:

  • Using a custom kernel on Arch
  • Turning off the sandboxing on Arch

I see a few long term options:

  • Figure out how to make Chrome/Chromium fail more elegantly in this
    scenario and contribute a patch.
  • Get user namespaces enabled in the Arch kernel -
    https://bugs.archlinux.org/task/36969 - might take a year until things
    are more settled down and they are comfortable with the security of the
    feature.
  • I don't know if this is possible, and it's a kernel change, but
    maybe allow network/pid namespacing without CAP_SYS_ADMIN.


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@xcellardoor

I'll have a pop at Zen Kernel, because why not?!

If not yeah I'll have to build my own Kernel. That will be a nice throwback to Gentoo :|

FYI you do NOT want to see what your pulseaudio build does when I try to run it for Skype as per your blog... I might open up an issue over there sometime. It's probably caused again by running on Arch.

@jessfraz
Owner
jessfraz commented Oct 7, 2015

Making your own package on arch for a kernel w a different config could be
really cool, I hear they have nice packaging tools obviously debians are
better ;) but yeah

On Wednesday, October 7, 2015, xcellardoor notifications@github.com wrote:

I'll have a pop at Zen Kernel, because why not?!

If not yeah I'll have to build my own Kernel. That will be a nice
throwback to Gentoo :|

FYI you do NOT want to see what your pulseaudio build does when I try to
run it for Skype as per your blog... I might open up an issue over there
sometime. It's probably caused again by running on Arch.


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@xcellardoor

Already done that, very easy :)
Yes the PKGBUILD system is very easy to use, and I remember using the Debian tools when I did a few packages for Ubuntu. Definitely prefer Arch's system :P You should try it sometime.

@xcellardoor

Failed with ZenKernel by the way... Next step is probably compiling my own.

@Kolbasz12

any luck with this lately?

@vikstrous

Nah, I'm planning to fork the AUFS kernel in the AUR and add user namespaces and call it the "docker" kernel. The only thing stopping me is that I need an easy way to make nvidia packages for it too and that's more effort. I'm hoping someone else will do it first.

@Kolbasz12

Can't we just pass puid and pgid like it is in some other containers? Or is this much more than that?

On Fri, Nov 13, 2015 at 7:42 PM -0800, "Viktor Stanchev" notifications@github.com wrote:

Nah, I'm planning to fork the AUFS kernel in the AUR and add user namespaces and call it the "docker" kernel. The only thing stopping me is that I need an easy way to make nvidia packages for it too and that's more effort. I'm hoping someone else will do it first.


Reply to this email directly or view it on GitHub.

@vikstrous

Chrome wants to do its own namespacing within the container, but it doesn't have privileges to do that for obvious reasons. User namespaces allow it create its own namespaces even though it's running within Docker. You have to completely disable Chrome's sandbox if you want it to run within Docker without user namespaces support. Since Chrome needs x11 access, it's not a good idea to disable its sandbox. With the sandbox disabled it's easier to compromise the Chrome within the container and if that happens it can use the x11 access to do keylogging, screenshots, keyboard emulation and other bad things. Running chrome as non-root within the container doesn't make a difference in this case.

@Kolbasz12

What is it about arch that makes this so much more difficult/complex vs the
likes of jess' favorite Debian?

On Sat, Nov 14, 2015, 12:07 AM Viktor Stanchev notifications@github.com
wrote:

Chrome wants to do its own namespacing within the container, but it
doesn't have privileges to do that for obvious reasons. User namespaces
allow it create its own namespaces even though it's running within Docker.
You have to completely disable Chrome's sandbox if you want it to run
within Docker without user namespaces support. Since Chrome needs x11
access, it's not a good idea to disable its sandbox. With the sandbox
disabled it's easier to compromise the Chrome within the container and if
that happens it can use the x11 access to do keylogging, screenshots,
keyboard emulation and other bad things. Running chrome as non-root within
the container doesn't make a difference in this case.


Reply to this email directly or view it on GitHub
#65 (comment)
.

@jessfraz
Owner

The kernel config, honestly compiling your own kernel is the easiest solution here you can even do it in a container

@Kolbasz12

Ha, if I only knew how... Time to Google

On Sat, Nov 14, 2015, 11:10 AM Jess Frazelle notifications@github.com
wrote:

The kernel config, honestly compiling your own kernel is the easiest
solution here you can even do it in a container


Reply to this email directly or view it on GitHub
#65 (comment)
.

@xcellardoor

Hi again,

I'm fine with building my own Kernel - that's easy! ;) but could someone tell me the key functionalities I have to ensure are built into the Kernel? Then I'll compile and report back :)

@xcellardoor

FYI by key functionalities I meant the ones Docker needs to do it's thing!

@hurricanehrndz

Check the docker source code, the check config is there under contrib, if I
remember correctly.
On Nov 15, 2015 7:49 AM, "Sam Cater" notifications@github.com wrote:

FYI by key functionalities I meant the ones Docker needs to do it's thing!


Reply to this email directly or view it on GitHub
#65 (comment)
.

@xcellardoor

Thanks @hurricanehrndz - I found https://github.com/docker/docker/blob/master/contrib/check-config.sh - and from that I believe that the following are the flags I need to set, which I shall now do. The big one which stood out to me was 'USER_NS'.

USER_NS
EXT3_FS EXT3_FS_XATTR EXT3_FS_POSIX_ACL EXT3_FS_SECURITY
EXT4_FS EXT4_FS_POSIX_ACL EXT4_FS_SECURITY
AUFS_FS
BTRFS_FS
BLK_DEV_DM DM_THIN_PROVISIONING
OVERLAY_FS

It's make time!

@xcellardoor

My messy screenshot shows we have liftoff.

http://i.imgur.com/dadMx4m.png

In the end the key change to the Kernel was USER_NS. I also added some EXT3_FS_SECURITY that wasn't enabled before and a couple of other options, but USER_NS was the big one. With that support, launching Chrome through Docker works just fine.

So what is the best way to proceed... Post my config.gz to a Git repo for people to use when building their own Kernel? (That would go out of date quickly though). A blog post with the instructions? Let me know your thoughts.

@xcellardoor

Job done with a blog post - https://blog.samcater.com/docker-arch-linux-and-user-namespaces/

Jess I guess you can close the issue now. Thanks everyone for the help in working out what needs to be done.

@CharlieKuharski

Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted

I'm getting the same issue.
It appears on docker-machine AND kitematic.
OS: Wins 10, VirtualBox.

@jessfraz
Owner
jessfraz commented Mar 3, 2016

It's not exactly an issue read above

On Thursday, March 3, 2016, Charlie Kuharski notifications@github.com
wrote:

Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted

I'm getting the same issue.

It appears on docker-machine AND kitematic.
OS: Wins 10, VirtualBox.


Reply to this email directly or view it on GitHub
#65 (comment)
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@anshprat

Hi Jess,

I would just like to append a bit more info here since I started facing this issue yday after a docker update.
A bit of usage context - till yesterday, I was using default fedora docker (1.9) on Fedora 23

Linux hostname 4.4.2-301.fc23.x86_64 #1 SMP Tue Feb 23 19:00:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

And chrome docker was working to my satisfaction.

Y'day, for using latest docker-compose features, I ended up updating my docker to 1.10 from docker repo.

After updating to above docker, my chrome docker is failing with..

[anshup@mouthwa ~]$ docker run -it --net host --cpuset-cpus 0 --memory 512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY -v $HOME/Downloads:/root/Downloads -v $HOME/.config/google-chrome/:/data --rm -v /dev/shm:/dev/shm jess/chrome
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted

So, there was no change in kernel, but an update in docker version. Am on F23.

[anshup@mouthwa ~]$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: Fedora
Description: Fedora release 23 (Twenty Three)
Release: 23
Codename: TwentyThree

I then updated my kernel to latest, and I still ve the same issue. I am thinking this is a docker related issue rather than kernel issue. Thoughts?

[anshup@mouthwa ~]$ uname -a
Linux hostname 4.4.5-300.fc23.x86_64 #1 SMP Thu Mar 10 17:54:44 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

@jessfraz
Owner

Use my custom seccomp profile in my dot files repo

On Thursday, March 17, 2016, Anshu Prateek notifications@github.com wrote:

Hi Jess,

I would just like to append a bit more info here since I started facing
this issue yday after a docker update.
A bit of usage context - till yesterday, I was using default fedora docker
(1.9) on Fedora 23

Linux hostname 4.4.2-301.fc23.x86_64 #1
#1 SMP Tue Feb 23 19:00:38
UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

And chrome docker was working to my satisfaction.

Y'day, for using latest docker-compose features, I ended up updating my
docker to 1.10 from docker repo.

After updating to above docker, my chrome docker is failing with..

[anshup@mouthwa ~]$ docker run -it --net host --cpuset-cpus 0 --memory
512mb -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY -v
$HOME/Downloads:/root/Downloads -v $HOME/.config/google-chrome/:/data --rm
-v /dev/shm:/dev/shm jess/chrome
Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted

So, there was no change in kernel, but an update in docker version. Am on
F23.

[anshup@mouthwa ~]$ lsb_release -a
LSB Version:
:core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: Fedora
Description: Fedora release 23 (Twenty Three)
Release: 23
Codename: TwentyThree

I then updated my kernel to latest, and I still ve the same issue. I am
thinking this is a docker related issue rather than kernel issue. Thoughts?

[anshup@mouthwa ~]$ uname -a
Linux hostname 4.4.5-300.fc23.x86_64 #1
#1 SMP Thu Mar 10 17:54:44
UTC 2016 x86_64 x86_64 x86_64 GNU/Linux


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@anshprat

@jfrazelle awesome, it worked, thanks :)

@bmustiata

@jfrazelle this is amazing. How did you managed to build that out?
Unreal. Fantastic work.

@thiagorider
thiagorider commented May 4, 2016 edited

I've got the same in Ubuntu 16.04 with Docker v1.11:

Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted

@jessfraz
Owner
jessfraz commented May 5, 2016

you need user namespaces

On Wed, May 4, 2016 at 2:08 PM, Thiago Rider Augusto <
notifications@github.com> wrote:

I've got the same in Ubuntu 16.04 with Docker v1.11:

Failed to move to new namespace: PID namespaces supported, Network
namespace supported, but failed: errno = Operation not permitted


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@xcellardoor

@thiagorider If Ubuntu 16.04 doesn't have User Namespaces enabled in the Kernel then you're going to have to compile and install your own I'm afraid. I had this problem in Arch and wrote the basics of what you have to enable in the Kernel for this to work

@thiagorider
thiagorider commented May 5, 2016 edited

@xcellardoor @jfrazelle When I was using Ubuntu 14.04 and Docker 1.9 it worked. I should find where to enable user namespaces in my kernel. It is different than Arch :-(
Was this option enabled in past kernels by default? Or my past Docker version(1.9) didn't required it?

@jessfraz
Owner
jessfraz commented May 5, 2016

It's chrome that needs it for the sandbox

On Thursday, May 5, 2016, Thiago Rider Augusto notifications@github.com
wrote:

@xcellardoor https://github.com/xcellardoor @jfrazelle
https://github.com/jfrazelle When I was using Ubuntu 14.04 and Docker
1.10
it worked. I should find where to enable user namespaces in my
kernel. It is different than Arch :-(
Was this option enabled in past kernels by default? Or my past Docker
version(1.9) didn't required it?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@thiagorider

@jfrazelle Is it related to the last Chrome release?

@jessfraz
Owner
jessfraz commented May 5, 2016

It's been in chrome for awhile now

On Thursday, May 5, 2016, Thiago Rider Augusto notifications@github.com
wrote:

@jfrazelle https://github.com/jfrazelle Is it related to the last
Chrome release?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#65 (comment)

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@bmustiata

@thiagorider you need to run it with:

docker run ... --security-opt seccomp:/path/to/chrome.json ...

The chrome.json file you can get it from: https://raw.githubusercontent.com/jfrazelle/dotfiles/master/etc/docker/seccomp/chrome.json

It seems to grant special privileges to the container. I am also using Ubuntu 16.04, and Docker 1.11, and there is no need to recompile the kernel.

@thiagorider

@bmustiata Thank you!

@cdenneen

How to get sound to work with Kitematic on Mac:

for those trying to get this working I've used the following to launch chrome successfully but can't get sound working as of yet... hopefully soon with someone's help here:

docker run -d \
        --memory 3gb \
        --net host \
        -v /etc/localtime:/etc/localtime:ro \
        -e DISPLAY=192.168.99.1:0 \
        -v $HOME/Downloads:/root/Downloads \
        -v $HOME/Pictures:/root/Pictures \
        -v $HOME/Torrents:/root/Torrents \
        -v $HOME/.chrome:/data \
        -v /dev/shm:/dev/shm \
        -v /etc/hosts:/etc/hosts \
        --security-opt seccomp:/path/to/chrome.json \
        --group-add audio \
        --group-add video \
        --name chrome \
        jess/chrome --user-data-dir=/data --force-device-scale-factor=1

Also used this:
docker/docker#8710 (comment)

Also another oddity is browsing works fine but the "Sign-in" button just "spins"

@jessfraz
Owner

Closing as needed to compile a new kernel

@jessfraz jessfraz closed this Jun 13, 2016
@WhisperingChaos

@jfrazelle

Just wanted to thank you for your effort to unleash containers on the Desktop! I used this thread to help solve a problem with another Docker-Chrome project.

@nunobaba

So far, creating a new user and use it to launch Chromium is a working workaround on Arch. Except that for some reasons I haven't pinpointed out yet, without the --privileged flag the same error message "Failed to move to new namespace..." keeps appearing. Which devices Chromium needs to start? I'm wondering it.

@jessfraz
Owner

It's because arch doesnot have user namespaces enabled.

On Mon, Aug 22, 2016 at 1:15 PM, Thuan Hu notifications@github.com wrote:

So far, creating a new user and use it to launch Chromium is a working
workaround on Arch. Except that for some reasons I haven't pinpointed out
yet, without the --privileged flag the same error message "Failed to move
to new namespace..." keeps appearing. Which devices Chromium needs to
start? I'm wondering it.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#65 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABYNbBjzDBJ9NyVC6bVo5rPi9YsrVCxkks5qigNPgaJpZM4GCdXL
.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@nunobaba

Confirmed. The ticket says it all. Thanks @jfrazelle for this lightning fast answer.

@chinthakagodawita chinthakagodawita referenced this issue in Sitback/docker-containers Nov 2, 2016
Merged

Fix issue with running Chrome under Ubuntu 16.04 #57

@NodeGuy
NodeGuy commented Dec 12, 2016

Thank you Jessie for an inspiring lead.

I'm excited to migrate from Mac OS to containerized Linux but I'm stuck at this step.

I'm getting the following error message when I try to run Google Chrome in the container:

# google-chrome
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
Illegal instruction (core dumped)

My host is Debian Sid running in VirtualBox on Mac OS. I believe that NS_USER is enabled by default in the kernel already:

$ grep USER_NS /boot/config-$(uname -r)
CONFIG_USER_NS=y

I'm disabling seccomp in my docker run:

docker run \
  --env DISPLAY \
  --interactive \
  --rm \
  --security-opt seccomp=unconfined \
  --tty \
  --volume /tmp/.X11-unix:/tmp/.X11-unix \
  my-debian

Here's what strace shows (but I don't know how to interpret it):

access("/opt/google/chrome/chrome-sandbox", F_OK) = 0
stat("/opt/google/chrome/chrome-sandbox", {st_mode=S_IFREG|S_ISUID|0755, st_size=14464, ...}) = 0
access("/opt/google/chrome/chrome-sandbox", X_OK) = 0
pipe([12, 13])                          = 0
close(13)                               = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f58d1ecad90) = 43
rt_sigprocmask(SIG_SETMASK, [], ~[KILL STOP RTMIN RT_1], 8) = 0
close(12)                               = 0
close(11)                               = 0
recvmsg(10, Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
{msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="", iov_len=13}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 0
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x5605fae8bd9b} ---
+++ killed by SIGILL (core dumped) +++
Illegal instruction

I'm keeping my Dockerfile simple:

FROM debian:sid
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get upgrade --yes

RUN apt-get install --yes \
  wget

RUN wget \
  https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb

RUN dpkg --install google-chrome-stable_current_amd64.deb; \
  apt-get install --fix-broken --yes

Chrome works fine outside of the container.

Does anyone have any thoughts?

@jessfraz
Owner
@NodeGuy
NodeGuy commented Dec 12, 2016

Thanks for the tip, that fixed the problem!

Here's an article I found describing the background of why it's not enabled by default: Controlling access to user namespaces and here's how to enable it (from Enable user namespaces in Debian kernel):

echo 'kernel.unprivileged_userns_clone=1' > /etc/sysctl.d/00-local-userns.conf
service procps restart
@mastermindg

My host OS is Ubuntu 16.10. I'm getting the namespace exception when I attempt to run jess/chrome. I've set the kernel parameter both in the container and outside and have restarted procps but I'm still getting the exception.

@RobCherry RobCherry referenced this issue in RobCherry/docker-chromedriver Dec 19, 2016
Open

"Chrome failed to start" #7

@cristianprice

I know this might be late but running the container with --privileged option and CMD "google-chrome", "--no-sandbox" , "--user-data-dir" solves the issue on arch linux.
My version: Linux version 4.8.13-1-ARCH (builduser@tobias) (gcc version 6.2.1 20160830 (GCC) ) #1 SMP PREEMPT Fri Dec 9 07:24:34 CET 2016
I do NOT have user namespaces enabled:
lxc-checkpoint prints :

Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: missing
Network namespace: enabled

@jsosic
jsosic commented Jan 7, 2017

Sorry to jump into the party, but I've been hit with this issue with latest CoreOS and docker 1.12.x. This worked perfectly with CoreOS shipping docker 1.11.x:

[root@13ab34c36c82 /]# /opt/google/chrome/chrome
Failed to move to new namespace: PID namespaces supported,
  Network namespace supported,
  but failed: errno = Operation not permitted
Aborted (core dumped)

Now, I have to either run the container with --cap-add=SYS_ADMIN or --privileged. More on this link:

http://serverfault.com/questions/824809/chrome-under-docker-cap-sys-admin-vs-privileged

@xcellardoor

@jsosic run this on the CoreOS box - zgrep USER_NS /proc/config.gz

Does it come back with CONFIG_USER_NS=y or CONFIG_USER_NS=n ?

@imkarrer
imkarrer commented Jan 12, 2017 edited

I am also having this issue on CoreOS. I tried using the --security-opt seccomp:/path/to/chrome.json and it is complaining about not opening display. I need to figure out how to update my ssh_config to enable X11Forwarding to see what happens next.

This is my error after using the --security-opt which gets past the namespacing issue. I think once I enable X11 I will be good, but there is no editor on my docker image to make this quick. Will update this post when I edit ssh_config.
[7:7:0112/183421:ERROR:browser_main_loop.cc(271)] Gtk: cannot open display:

@xcellardoor what output are you expecting from zgrep USER_NS /proc/config.gz? I assume it is CONFIG_USER_NS=y.

@jsosic how did exec_linux.go help you? Where did you execute it? On the docker container or the coreos machine hosting the docker engine?

edit: nevermind, --cap-add=SYS_ADMIN worked for me, thanks @jsosic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment