This repository has been archived by the owner on Jan 10, 2023. It is now read-only.
Nested containers [WIP] #91
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sargun
force-pushed
the
nested-containers-2
branch
from
807dfa3
to
7f60368
Compare
sargun
force-pushed
the
nested-containers-2
branch
2 times, most recently
from
b50c70d
to
01242a2
Compare
|
This is critical for MCE, bionic gluing, and for "pods" |
|
@fabiokung, @andrew-leung I'm requesting a very early, review here. |
fabiokung
reviewed
May 2, 2018
| hostCfg.SecurityOpt = append(hostCfg.SecurityOpt, fmt.Sprintf("seccomp=%s", string(seccomp.MustAsset("nested-container.json")))) | ||
|
|
||
| if _, ok := addedCapabilities["SYS_ADMIN"]; !ok { | ||
| hostCfg.CapAdd = append(hostCfg.CapAdd, "SYS_ADMIN") |
There was a problem hiding this comment.
lol. I'll unfuck this. I promise. See my TODOs.
fabiokung
reviewed
May 2, 2018
| @@ -518,13 +500,10 @@ func (r *DockerRuntime) setupLogs(c *runtimeTypes.Container, containerCfg *conta | |||
| c.Env["TITUS_REDIRECT_STDOUT"] = "/logs/stdout" | |||
| c.Env["TITUS_UNIX_CB_PATH"] = filepath.Join("/titus-executor-sockets/", socketFileName) | |||
| /* Require us to send a message to tini in order to let it know we're ready for it to start the container */ | |||
| c.Env["TITUS_CONFIRM"] = "true" | |||
| c.Env["TITUS_CONFIRM"] = trueString | |||
There was a problem hiding this comment.
unrelated note here, @andrew-leung mentioned some of these titus-executor -> tini ENV vars were leaking into user entrypoints earlier this week. This may be a good opportunity to review them and unsetenv what's missing on tini.
fabiokung
reviewed
May 2, 2018
| for _, line := range strings.Split(string(cgroups), "\n") { | ||
| cgroupInfo := strings.Split(strings.TrimSpace(line), ":") | ||
| if len(cgroupInfo) != 3 { | ||
| continue |
There was a problem hiding this comment.
this copy was largely copied from existing libcontainer cgroup parsing code. This a poor man's way of avoiding cgroup2. I don't see a reason to log that?
fabiokung
reviewed
May 2, 2018
| } | ||
| controllerType := cgroupInfo[1] | ||
| if len(controllerType) == 0 { | ||
| continue |
There was a problem hiding this comment.
If you have lots of controller-less cgroups, you'll get a lot of not-useful warnings.
fabiokung
reviewed
May 2, 2018
| @@ -154,3 +159,35 @@ func cleanupCgroups(cgroupPath string) error { | |||
|
|
|||
| return nil | |||
| } | |||
|
|
|||
| func setupContainerNesting(parentCtx context.Context, c *runtimeTypes.Container, cred ucred) error { | |||
There was a problem hiding this comment.
maybe add some notes about cgroups v1 vs v2 support of the code below?
fabiokung
reviewed
May 2, 2018
| return nil | ||
| } | ||
|
|
||
| var _defaultJson = []byte("\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\xff\xe4\x5a\x5d\x4f\x1c\x3b\xd2\xbe\x1e\x7e\x05\x9a\xeb\x5c\x10\x42\x38\x24\x77\xf3\x12\xde\x4d\xb4\x21\x64\x81\xd5\x39\x47\xab\xc8\x32\xee\xea\x1e\xef\xf8\x0b\x97\x7b\x60\x14\xe5\xbf\xaf\xdc\x3d\xd3\x2e\xbb\xe7\x44\x99\x90\xc0\xae\x72\x01\xf2\xf3\xb8\x6c\x57\x95\xed\x72\xd9\x3d\x9f\xf7\x26\xd3\x0a\x6a\xde\xaa\x30\x13\x41\x5a\x33\x7d\xbd\x3f\xbd\x3a\x3d\xff\xc8\x66\xa7\xd7\xec\xec\xf2\xf2\xc3\xc5\xf4\xd9\xde\x64\xca\xbd\x98\x9f\x73\x37\x7d\xbd\xff\xaf\xbd\xc9\xe4\xf3\xde\x64\xd2\x73\x32\x80\x08\xad\x87\xd4\xec\xf2\xf4\x2d\xfb\xe3\xe4\x98\x1d\x1f\xc5\x86\x93\xc9\x14\xdb\x9b\x19\x91\xc4\x75\x1f\x93\x49\xde\xa0\x97\xce\xc8\x17\x87\xd3\xc8\x7d\xda\x9b\x4c\xbe\x3c\xfb\x86\x61\x67\xf1\xff\x4e\xe3\xce\x2e\xcf\x77\x1b\xe2\xfc\xdd\xc7\xab\x9d\x46\x88\x0d\xc6\xa6\xf5\xdd\x7c\xd8\xd5\xc0\xd4\xec\xc7\x28\xb0\xfb\xe8\x67\xef\x77\xb6\xfe\xec\xfd\xf6\xe1\x63\x57\xdf\xe5\x81\x4d\xc3\x1f\xa7\xc6\x6e\x3a\x5c\xbd\x78\x75\xf0\xc7\x0e\xa3\x47\xf9\x34\xc2\xde\xe4\x53\xdc\x50\xb8\x42\xc1\x95\xc2\x6c\x47\x19\xae\x69\x17\x5c\x08\x70\x61\xa3\x76\x8f\x8e\x28\x44\x1c\x50\xf5\xef\x20\x35\xdc\x0f\x58\x71\xaf\x37\xe0\x46\x9a\x6a\x28\xfb\xc5\xa6\x28\xb8\x6b\x20\x10\x84\x04\xcd\x2b\xe9\x13\xd0\xb6\x4a\xc0\xde\x99\x0c\x6c\xa6\x62\x32\x15\xca\x8a\x05\x6b\x20\x44\x4f\x94\x5c\xd4\x2f\x27\x0d\x37\x16\x15\x80\x23\x34\x26\x19\x6b\x0c\x88\xa4\x91\x75\x2b\x56\x4b\x05\xcc\x73\xd3\x24\x29\x0f\x7c\x90\xa9\x5a\x47\x8a\x87\xa4\xfc\x62\x53\x06\x67\x95\x62\x5d\x2b\xd8\xc6\x3d\x2f\xc8\xa0\x46\x04\xb3\xaa\xca\x49\x77\xc7\x65\xc8\xa9\xed\x4c\xd6\x74\x09\x26\xd4\x25\x1c\x94\x86\x7b\x10\x4b\xc8\x51\xb2\x14\xee\x65\x56\x66\x8d\xb7\xc9\xf8\xba\x5f\x1c\x49\xbc\xe6\xd5\x52\x22\x1c\x1f\x8d\x08\x46\x39\xa5\xac\x20\x7e\xa9\xb9\xb1\x41\xd6\x2b\xa6\x79\x5a\x35\x75\xb6\x34\xea\x6c\x6d\xf4\x88\x0c\x9b\x2d\x96\xba\x58\x2d\x3d\xa6\xd2\x26\x79\xbb\x03\x44\xb7\x8a\x07\x8e\x2b\x23\x06\xa2\x81\x70\xcf\x43\x48\x9a\x28\x89\x25\x63\x45\x52\xdb\x12\x13\x3c\x68\xbb\x84\x5c\x18\xcb\xfe\x30\x10\xd5\x22\x20\xda\x44\x38\x22\x6a\x2c\x20\xad\xa7\xaa\x07\xdf\x9a\xcc\xcf\x1b\x82\x34\x68\x43\xda\xcc\x75\x1b\xf7\x0e\x99\xce\x06\x82\x70\x2d\x45\x77\x15\x41\x15\x98\x80\x25\x4e\x7d\x37\x10\xa0\x91\x55\x01\xd3\xbc\x44\xa2\xcd\xeb\xdb\xa2\x3e\x6f\x5e\xb6\xee\xd6\x22\x8e\x88\x4c\x46\x46\x93\x3c\x21\x1c\x80\x8f\xf1\x8f\x52\xf9\x30\xae\xf1\x8e\xc2\xbc\xb2\x80\x5e\x5a\x2f\xc3\x8a\x50\x9e\x9b\xca\x6a\x4a\x00\xe6\x03\xf4\x44\xa6\xa6\x07\x6c\x4b\x99\xd2\x19\x5e\x49\x2d\xe9\xdc\x30\x6f\x6f\x5a\x0c\x2c\xae\x48\x2a\xd7\x22\x6f\xa8\x7d\x98\xf5\x8c\x56\x2c\x0a\x07\x44\xca\xba\xac\xeb\x30\xf7\xc0\x2b\xc6\x3d\x70\x42\x87\xac\xa7\xe8\x5a\x5b\x57\x9c\x5a\x9f\x5b\x51\x9a\x90\xad\x7c\xb9\xde\xf5\xbc\xaa\xd8\x1d\x0f\x62\x5e\x56\x48\x93\xcc\xa5\xdc\xf3\x92\xf4\xba\xe8\xc0\x32\xc1\x8d\x00\x95\x08\x12\x60\xa5\x65\x15\x60\xf0\x76\x45\x98\xb8\xfa\x96\x74\x3d\x4b\x1b\xe7\x96\x91\x93\x6b\xcd\x20\x65\x22\x4a\xf1\x30\xe2\xf6\x86\x4c\x92\x74\xc3\x6e\x5c\x48\x35\x68\xa0\xb2\x78\xa5\x8a\x78\xa5\x4a\x47\x29\x69\x16\xb4\x9c\x36\x68\x9c\x79\x30\x14\xe5\x0d\x47\x0c\x53\x0a\x01\x52\x6f\x5b\x22\x54\x2e\x50\x86\x2b\x45\xc3\x95\xca\xc3\x95\xee\x83\xfd\x00\x41\xd7\x55\x71\x04\x6a\x69\x84\xf5\x09\x2e\x48\x8c\xef\x40\xea\x5c\x2f\x4c\x8a\xf8\x1d\x20\x75\x34\xe4\x76\xe0\x30\x43\x3c\xf9\x5a\x6b\xee\x68\x39\x09\x3a\x6f\x03\x39\xf8\xf5\x6d\x9c\x6c\x84\x40\xcd\xd5\xb7\xac\x5f\x63\x84\xb0\x2e\xb9\x5c\xdf\xb2\xb8\x0b\x2a\x0f\x02\x64\x3a\x46\x37\x34\x42\xca\x88\xf4\x2d\x6b\x0d\x9d\x4a\xed\x81\xea\x86\x0d\x59\xa3\x1a\x1b\xb2\xf2\x34\x36\x5e\x2c\x09\x42\xd2\x2d\x0d\xf9\xba\x35\x99\x67\x7a\x48\xbd\xd1\x1a\x32\xe6\x28\x31\x32\x70\xb7\x3e\x73\x86\x05\x63\xe0\x0e\x41\x11\x3f\x51\xf3\x63\x39\xc9\x3a\xde\xa6\xd9\x77\xd2\x65\xe5\xc1\xef\x31\x49\x19\xca\x19\xf0\xc4\x01\x2e\x86\x9f\xb4\xb4\x3a\xb8\xcc\x51\xea\xb1\x8f\x8c\x44\xba\xd7\x78\xb8\xeb\xb9\x3b\x2f\xe9\xa9\xd7\xe3\x65\x01\x87\xfe\x62\xef\xb4\xcc\xe7\x05\x41\x67\x71\x83\x93\x1b\x32\x55\x3d\x88\xac\x5c\xfb\x74\x36\x44\xac\x35\x36\x19\xa6\x50\x73\xd7\xa7\xa2\x8e\x37\x29\xd3\xdd\xb2\x67\x3d\xd0\x88\xde\x23\xaa\x50\x8f\x89\x81\x18\xb8\x0f\x6c\x7d\x39\x18\x68\x4d\x76\x63\xac\x96\x0d\xef\x2f\xeb\x19\xe7\xc0\x54\xd2\x34\x05\xe9\xad\xd0\x1c\x17\x39\x7b\xdb\x42\x0b\xd2\xd4\x36\xa7\x3d\x84\xd6\x17\xbd\x62\x8b\x8e\xec\x96\x9e\xec\x36\x11\x4d\x71\x7d\x60\xa1\xd9\xd6\x31\x8a\x39\x54\x71\x03\xf3\xba\x8e\x67\xc4\x6a\x5c\x41\xfc\x35\x90\x8e\x7b\xae\x47\x2c\xdb\x1c\xeb\x4c\xf3\xfb\xaf\xd5\x4a\x33\xaa\xed\x0a\xad\x82\x62\x2c\xef\xbb\xa6\xd2\x04\xf0\x4b\xae\xf2\x4a\xfc\x2b\xb5\x71\x9b\xda\xb8\x55\x6d\xfc\xcb\xa1\x57\x12\xd2\x6d\x00\x41\x08\xab\x5d\x82\x74\x6b\x23\x68\xb2\x05\x11\x34\x89\x41\x08\xda\x92\x66\xba\x9b\x1a\xca\x98\x8a\x96\xe3\xb2\x2d\x71\xda\x80\x91\xa1\x2b\xbf\xc3\x39\x0c\x69\x6a\x21\xd4\x34\x83\xda\xe0\x74\x62\x76\x4c\x5b\x48\xb4\x85\x44\xde\x43\xd9\x3e\xcf\x25\x71\x9c\x4b\x62\x99\x4b\x62\x9e\x38\xe2\x38\x19\xc4\x98\xc6\xe5\x22\x1e\xca\x81\xf3\xec\x10\xc7\xd9\x21\x96\xd9\x21\x8e\xb3\xc3\x8e\x2a\x45\x46\x12\x59\xfe\x88\xdb\xf3\x47\xcc\xd2\x45\x1c\xe5\x86\xb8\x3d\x37\xec\x68\x59\xc5\x5c\xce\x93\xe7\x02\xcc\x12\x42\x2c\x12\xc2\x32\xb7\xc0\xb9\x4e\x41\x0b\xe7\xd9\x5a\x9c\xeb\x8a\x56\xd1\x85\x39\x6f\x43\x45\xd2\xa9\x18\xb5\x54\xc0\xc0\xd3\x39\x88\xb2\x31\x5c\xa5\x3b\xf0\x06\x1f\x11\x22\x8f\x49\xd1\x6a\x32\x44\x87\x68\xa8\xec\x19\xc7\x53\xbc\x44\xa7\xa4\x48\x4b\x9e\x1c\xa1\x79\x92\x94\xdf\xe0\xca\x0b\x1c\xae\x34\x3d\x5c\xd6\x90\xf4\x45\x0e\xfb\x58\xde\xf2\x52\x11\x69\x32\xc0\x0a\xb3\x30\xb9\x42\x65\x87\x7d\x16\x60\x68\x15\x1a\x9a\xa0\xd2\x37\x94\x6e\xd1\x17\x69\x5c\xcf\x55\xa0\xa0\xe0\x46\x09\xdf\x86\x2d\x1e\x66\x36\x34\x6e\xa1\xa3\xac\x5d\x82\xf7\xad\x19\xf1\x63\xe1\x2d\x3d\x0c\xc6\x87\xcc\xa6\xe2\x36\x3c\xbe\x0c\xb7\xa3\x4b\x56\x4b\x8f\xb4\x96\x9e\xb1\x79\x2a\xd7\xa3\x34\x4d\x2d\x55\xa9\x03\x06\x8b\xda\x41\xc9\x25\x7d\x32\x58\xea\x7c\x1d\xc5\xb3\xef\x88\x82\xb4\x9b\x22\x22\x97\xd1\x2e\x87\xc9\xc0\xb2\x7f\x0c\xec\xa8\x29\x1f\xbf\xba\xcf\xde\xbf\xbf\xf8\x7d\xba\xae\xf6\x4d\xf7\x20\xb8\x96\x16\x56\x6b\x30\x21\x8a\xaf\x05\xa4\x11\xaa\xad\xba\x57\xc3\xcf\x5f\x7a\x0a\xee\x09\x95\x3f\x6a\x16\x0f\x8c\x0e\x3c\x5a\xc3\x55\x8c\x8d\xbb\xeb\xd4\xf5\xd1\xf5\xdb\xe9\x51\xc1\xfd\xf4\xf5\xfe\xc1\xb3\x35\xb1\xe4\xaa\x85\x11\x71\x7d\x67\x29\x67\xdd\x30\x4a\xfc\x3b\xfb\x47\xa7\xc6\xe4\x0b\x51\xe6\x7f\xd0\xe4\x93\x5f\xcf\xe4\xe7\x2f\x9e\x1f\xfc\x76\xf8\x6b\xda\x7d\xf2\x0b\x2e\xf1\xa3\xc3\x57\x47\xaf\x8e\x7f\x3b\x7c\xf5\xf2\xbf\xcb\xf6\xe2\xec\x3d\xfc\x59\xa1\xb6\x1f\x8d\x7b\x31\x27\xa3\xc7\x7b\xb3\x38\x3e\x52\xd0\x9b\xf9\x29\xfe\xdf\xd9\x02\xee\x35\xdb\xf6\x3a\x1f\xf9\xaf\x64\x16\x99\xd1\x6b\xfe\xc6\x03\x5f\x38\x2b\x0d\xf9\xc8\x23\xe6\x50\xab\x16\xe7\x59\x82\xa8\xf0\x91\xfd\x94\x3e\x4e\x75\xe5\xf5\x17\xb8\xef\xf6\x98\x98\xb3\xfe\x91\xe2\x91\xad\xd0\xe9\x25\x64\x32\xbd\x5f\x7f\xcb\xfc\x4e\x2b\xb4\xad\x64\xbd\x62\xaa\x0a\x4f\x6e\xc5\x50\x3e\x39\x7e\x88\x45\xf8\xe2\xd5\x01\x73\x42\x32\xad\xa5\x65\xf4\xf1\x26\xaf\xc9\x32\xa4\xae\xca\xb7\x26\xa6\x62\x4c\x1a\x0c\xfe\x91\xdd\x11\x15\x18\x3c\x10\xc1\xfd\x43\x7c\x60\x1d\x18\x76\xb3\x62\x73\x6e\x2a\x05\x8c\xff\xe4\xc9\x15\xdc\x51\x5b\x4e\x67\x1f\xd9\x9b\xd9\x29\xbb\x3c\x9b\xbd\x61\x57\x67\xb3\xcb\xd3\xb7\x0f\x31\xe6\xc6\xd5\x43\x20\x51\xd6\x8c\xbf\x13\xd2\x0f\x03\xca\xda\x45\xeb\x58\x25\xac\x5d\xc8\xf4\x00\x6b\xdb\x14\x8d\x62\xf7\x2c\x58\xe2\x9c\x67\xc3\x09\x56\xb3\xee\xd9\x3f\x7b\xd4\xbd\x6d\x6d\xe0\xd9\x43\x48\xa8\xac\xe6\xd2\xd0\x0b\x00\x42\x98\x5b\x0c\x05\x65\x30\x5d\x1d\xa8\x0a\x3d\x3a\x4c\xf7\x05\x9c\x73\x0f\x8f\x3e\x49\x57\x7f\x5e\xb1\xd9\x9b\xf3\x77\x1f\x1e\x32\x3d\xfd\x9c\xfc\xac\x13\xff\xf0\xe0\xe4\xe0\xe5\xc1\xcb\x93\x97\xc7\x3b\x9c\xf8\xe7\xb3\xab\xbf\x9f\xbd\xf9\x31\x07\xff\x37\x3b\xef\xd9\x77\xed\xec\xc7\x73\xed\xf3\x47\x74\x6d\x34\x73\xbf\x7b\x95\x84\x00\x7e\xdf\xfa\x0a\xbc\x34\xcd\x7e\x6d\xfd\x7e\x67\xd4\xbe\xc4\xfd\x4a\xd6\x35\x78\xd8\xec\x8a\x9f\x11\x25\x77\x5b\xf9\x5f\x9d\x0b\x0f\x37\xd6\x3e\x7e\x1c\x8d\x8a\xfe\xdf\xc5\xc5\xf5\x83\x76\xe8\xdc\x3f\x95\xea\xa7\x6f\x2f\x1f\xa8\x7c\xff\xaa\xc4\xb4\xad\xda\xf4\x84\x1c\x23\x7e\x41\xd5\x5b\xb8\xdb\x16\xfc\x6a\xc3\x3d\x85\xf9\xe7\x17\x6f\xfe\xf9\xfe\xec\x41\x59\xa6\x10\x4f\x33\x73\x1f\x67\xa7\xa7\x0f\x9a\xb8\x85\x48\xdf\x15\x9c\xb7\x02\x10\xd9\x52\xb3\xe2\xcb\xdd\xc0\x17\xdf\xe0\x82\xe7\xe2\x69\xe6\xec\xe3\xf5\xe5\xec\xf4\x41\x73\x26\xad\x23\xbf\x2e\x70\xe0\xf5\x93\x18\x72\x39\xfb\xfd\xdd\xc5\x83\x32\xe9\x2d\xbf\xe9\xc0\xf1\xef\xe9\x36\x4f\xae\x4f\x61\xe3\xf5\xbb\xf3\x07\x4d\xd5\x72\xce\x4d\xd3\xba\xa7\xd1\xfd\xfa\x4f\x76\x7a\xf1\xe1\xff\xdf\xfd\xed\x1b\x2c\xd8\x9b\x7c\xda\xfb\xf2\x9f\x00\x00\x00\xff\xff\x9b\x8d\xde\xf4\x37\x2d\x00\x00") |
There was a problem hiding this comment.
I'm going to add details in: #97
fabiokung
reviewed
May 2, 2018
| return a, nil | ||
| } | ||
|
|
||
| var _nestedContainerJson = []byte("\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\xff\xe4\x5a\x5d\x4f\x1c\x3b\xd2\xbe\x1e\x7e\x05\x9a\xeb\x5c\x10\x42\x38\x24\x77\xf3\x12\xde\x4d\xb4\x21\x64\x81\xd5\x39\x47\xab\xc8\x32\xee\xea\x1e\xef\xf8\x0b\x97\x7b\x60\x14\xe5\xbf\xaf\xdc\x3d\xd3\x2e\xbb\xe7\x44\x99\x90\xc0\xae\x72\x01\xf2\xf3\xb8\x6c\x57\x95\xed\x72\xd9\x3d\x9f\xf7\x26\xd3\x0a\x6a\xde\xaa\x30\x13\x41\x5a\x33\x7d\xbd\x3f\xbd\x3a\x3d\xff\xc8\x66\xa7\xd7\xec\xec\xf2\xf2\xc3\xc5\xf4\xd9\xde\x64\xca\xbd\x98\x9f\x73\x37\x7d\xbd\xff\xaf\xbd\xc9\xe4\xf3\xde\x64\xd2\x73\x32\x80\x08\xad\x87\xd4\xec\xf2\xf4\x2d\xfb\xe3\xe4\x98\x1d\x1f\xc5\x86\x93\xc9\x14\xdb\x9b\x19\x91\xc4\x75\x1f\x93\x49\xde\xa0\x97\xce\xc8\x17\x87\xd3\xc8\x7d\xda\x9b\x4c\xbe\x3c\xfb\x86\x61\x67\xf1\xff\x4e\xe3\xce\x2e\xcf\x77\x1b\xe2\xfc\xdd\xc7\xab\x9d\x46\x88\x0d\xc6\xa6\xf5\xdd\x7c\xd8\xd5\xc0\xd4\xec\xc7\x28\xb0\xfb\xe8\x67\xef\x77\xb6\xfe\xec\xfd\xf6\xe1\x63\x57\xdf\xe5\x81\x4d\xc3\x1f\xa7\xc6\x6e\x3a\x5c\xbd\x78\x75\xf0\xc7\x0e\xa3\x47\xf9\x34\xc2\xde\xe4\x53\xdc\x50\xb8\x42\xc1\x95\xc2\x6c\x47\x19\xae\x69\x17\x5c\x08\x70\x61\xa3\x76\x8f\x8e\x28\x44\x1c\x50\xf5\xef\x20\x35\xdc\x0f\x58\x71\xaf\x37\xe0\x46\x9a\x6a\x28\xfb\xc5\xa6\x28\xb8\x6b\x20\x10\x84\x04\xcd\x2b\xe9\x13\xd0\xb6\x4a\xc0\xde\x99\x0c\x6c\xa6\x62\x32\x15\xca\x8a\x05\x6b\x20\x44\x4f\x94\x5c\xd4\x2f\x27\x0d\x37\x16\x15\x80\x23\x34\x26\x19\x6b\x0c\x88\xa4\x91\x75\x2b\x56\x4b\x05\xcc\x73\xd3\x24\x29\x0f\x7c\x90\xa9\x5a\x47\x8a\x87\xa4\xfc\x62\x53\x06\x67\x95\x62\x5d\x2b\xd8\xc6\x3d\x2f\xc8\xa0\x46\x04\xb3\xaa\xca\x49\x77\xc7\x65\xc8\xa9\xed\x4c\xd6\x74\x09\x26\xd4\x25\x1c\x94\x86\x7b\x10\x4b\xc8\x51\xb2\x14\xee\x65\x56\x66\x8d\xb7\xc9\xf8\xba\x5f\x1c\x49\xbc\xe6\xd5\x52\x22\x1c\x1f\x8d\x08\x46\x39\xa5\xac\x20\x7e\xa9\xb9\xb1\x41\xd6\x2b\xa6\x79\x5a\x35\x75\xb6\x34\xea\x6c\x6d\xf4\x88\x0c\x9b\x2d\x96\xba\x58\x2d\x3d\xa6\xd2\x26\x79\xbb\x03\x44\xb7\x8a\x07\x8e\x2b\x23\x06\xa2\x81\x70\xcf\x43\x48\x9a\x28\x89\x25\x63\x45\x52\xdb\x12\x13\x3c\x68\xbb\x84\x5c\x18\xcb\xfe\x30\x10\xd5\x22\x20\xda\x44\x38\x22\x6a\x2c\x20\xad\xa7\xaa\x07\xdf\x9a\xcc\xcf\x1b\x82\x34\x68\x43\xda\xcc\x75\x1b\xf7\x0e\x99\xce\x06\x82\x70\x2d\x45\x77\x15\x41\x15\x98\x80\x25\x4e\x7d\x37\x10\xa0\x91\x55\x01\xd3\xbc\x44\xa2\xcd\xeb\xdb\xa2\x3e\x6f\x5e\xb6\xee\xd6\x22\x8e\x88\x4c\x46\x46\x93\x3c\x21\x1c\x80\x8f\xf1\x8f\x52\xf9\x30\xae\xf1\x8e\xc2\xbc\xb2\x80\x5e\x5a\x2f\xc3\x8a\x50\x9e\x9b\xca\x6a\x4a\x00\xe6\x03\xf4\x44\xa6\xa6\x07\x6c\x4b\x99\xd2\x19\x5e\x49\x2d\xe9\xdc\x30\x6f\x6f\x5a\x0c\x2c\xae\x48\x2a\xd7\x22\x6f\xa8\x7d\x98\xf5\x8c\x56\x2c\x0a\x07\x44\xca\xba\xac\xeb\x30\xf7\xc0\x2b\xc6\x3d\x70\x42\x87\xac\xa7\xe8\x5a\x5b\x57\x9c\x5a\x9f\x5b\x51\x9a\x90\xad\x7c\xb9\xde\xf5\xbc\xaa\xd8\x1d\x0f\x62\x5e\x56\x48\x93\xcc\xa5\xdc\xf3\x92\xf4\xba\xe8\xc0\x32\xc1\x8d\x00\x95\x08\x12\x60\xa5\x65\x15\x60\xf0\x76\x45\x98\xb8\xfa\x96\x74\x3d\x4b\x1b\xe7\x96\x91\x93\x6b\xcd\x20\x65\x22\x4a\xf1\x30\xe2\xf6\x86\x4c\x92\x74\xc3\x6e\x5c\x48\x35\x68\xa0\xb2\x78\xa5\x8a\x78\xa5\x4a\x47\x29\x69\x16\xb4\x9c\x36\x68\x9c\x79\x30\x14\xe5\x0d\x47\x0c\x53\x0a\x01\x52\x6f\x5b\x22\x54\x2e\x50\x86\x2b\x45\xc3\x95\xca\xc3\x95\xee\x83\xfd\x00\x41\xd7\x55\x71\x04\x6a\x69\x84\xf5\x09\x2e\x48\x8c\xef\x40\xea\x5c\x2f\x4c\x8a\xf8\x1d\x20\x75\x34\xe4\x76\xe0\x30\x43\x3c\xf9\x5a\x6b\xee\x68\x39\x09\x3a\x6f\x03\x39\xf8\xf5\x6d\x9c\x6c\x84\x40\xcd\xd5\xb7\xac\x5f\x63\x84\xb0\x2e\xb9\x5c\xdf\xb2\xb8\x0b\x2a\x0f\x02\x64\x3a\x46\x37\x34\x42\xca\x88\xf4\x2d\x6b\x0d\x9d\x4a\xed\x81\xea\x86\x0d\x59\xa3\x1a\x1b\xb2\xf2\x34\x36\x5e\x2c\x09\x42\xd2\x2d\x0d\xf9\xba\x35\x99\x67\x7a\x48\xbd\xd1\x1a\x32\xe6\x28\x31\x32\x70\xb7\x3e\x73\x86\x05\x63\xe0\x0e\x41\x11\x3f\x51\xf3\x63\x39\xc9\x3a\xde\xa6\xd9\x77\xd2\x65\xe5\xc1\xef\x31\x49\x19\xca\x19\xf0\xc4\x01\x2e\x86\x9f\xb4\xb4\x3a\xb8\xcc\x51\xea\xb1\x8f\x8c\x44\xba\xd7\x78\xb8\xeb\xb9\x3b\x2f\xe9\xa9\xd7\xe3\x65\x01\x87\xfe\x62\xef\xb4\xcc\xe7\x05\x41\x67\x71\x83\x93\x1b\x32\x55\x3d\x88\xac\x5c\xfb\x74\x36\x44\xac\x35\x36\x19\xa6\x50\x73\xd7\xa7\xa2\x8e\x37\x29\xd3\xdd\xb2\x67\x3d\xd0\x88\xde\x23\xaa\x50\x8f\x89\x81\x18\xb8\x0f\x6c\x7d\x39\x18\x68\x4d\x76\x63\xac\x96\x0d\xef\x2f\xeb\x19\xe7\xc0\x54\xd2\x34\x05\xe9\xad\xd0\x1c\x17\x39\x7b\xdb\x42\x0b\xd2\xd4\x36\xa7\x3d\x84\xd6\x17\xbd\x62\x8b\x8e\xec\x96\x9e\xec\x36\x11\x4d\x71\x7d\x60\xa1\xd9\xd6\x31\x8a\x39\x54\x71\x03\xf3\xba\x8e\x67\xc4\x6a\x5c\x41\xfc\x35\x90\x8e\x7b\xae\x47\x2c\xdb\x1c\xeb\x4c\xf3\xfb\xaf\xd5\x4a\x33\xaa\xed\x0a\xad\x82\x62\x2c\xef\xbb\xa6\xd2\x04\xf0\x4b\xae\xf2\x4a\xfc\x2b\xb5\x71\x9b\xda\xb8\x55\x6d\xfc\xcb\xa1\x57\x12\xd2\x6d\x00\x41\x08\xab\x5d\x82\x74\x6b\x23\x68\xb2\x05\x11\x34\x89\x41\x08\xda\x92\x66\xba\x9b\x1a\xca\x98\x8a\x96\xe3\xb2\x2d\x71\xda\x80\x91\xa1\x2b\xbf\xc3\x39\x0c\x69\x6a\x21\xd4\x34\x83\xda\xe0\x74\x62\x76\x4c\x5b\x48\xb4\x85\x44\xde\x43\xd9\x3e\xcf\x25\x71\x9c\x4b\x62\x99\x4b\x62\x9e\x38\xe2\x38\x19\xc4\x98\xc6\xe5\x22\x1e\xca\x81\xf3\xec\x10\xc7\xd9\x21\x96\xd9\x21\x8e\xb3\xc3\x8e\x2a\x45\x46\x12\x59\xfe\x88\xdb\xf3\x47\xcc\xd2\x45\x1c\xe5\x86\xb8\x3d\x37\xec\x68\x59\xc5\x5c\xce\x93\xe7\x02\xcc\x12\x42\x2c\x12\xc2\x32\xb7\xc0\xb9\x4e\x41\x0b\xe7\xd9\x5a\x9c\xeb\x8a\x56\xd1\x85\x39\x6f\x43\x45\xd2\xa9\x18\xb5\x54\xc0\xc0\xd3\x39\x88\xb2\x31\x5c\xa5\x3b\xf0\x06\x1f\x11\x22\x8f\x49\xd1\x6a\x32\x44\x87\x68\xa8\xec\x19\xc7\x53\xbc\x44\xa7\xa4\x48\x4b\x9e\x1c\xa1\x79\x92\x94\xdf\xe0\xca\x0b\x1c\xae\x34\x3d\x5c\xd6\x90\xf4\x45\x0e\xfb\x58\xde\xf2\x52\x11\x69\x32\xc0\x0a\xb3\x30\xb9\x42\x65\x87\x7d\x16\x60\x68\x15\x1a\x9a\xa0\xd2\x37\x94\x6e\xd1\x17\x69\x5c\xcf\x55\xa0\xa0\xe0\x46\x09\xdf\x86\x2d\x1e\x66\x36\x34\x6e\xa1\xa3\xac\x5d\x82\xf7\xad\x19\xf1\x63\xe1\x2d\x3d\x0c\xc6\x87\xcc\xa6\xe2\x36\x3c\xbe\x0c\xb7\xa3\x4b\x56\x4b\x8f\xb4\x96\x9e\xb1\x79\x2a\xd7\xa3\x34\x4d\x2d\x55\xa9\x03\x06\x8b\xda\x41\xc9\x25\x7d\x32\x58\xea\x7c\x1d\xc5\xb3\xef\x88\x82\xb4\x9b\x22\x22\x97\xd1\x2e\x87\xc9\xc0\xb2\x7f\x0c\xec\xa8\x29\x1f\xbf\xba\xcf\xde\xbf\xbf\xf8\x7d\xba\xae\xf6\x4d\xf7\x20\xb8\x96\x16\x56\x6b\x30\x21\x8a\xaf\x05\xa4\x11\xaa\xad\xba\x57\xc3\xcf\x5f\x7a\x0a\xee\x09\x95\x3f\x6a\x16\x0f\x8c\x0e\x3c\x5a\xc3\x55\x8c\x8d\xbb\xeb\xd4\xf5\xd1\xf5\xdb\xe9\x51\xc1\xfd\xf4\xf5\xfe\xc1\xb3\x35\xb1\xe4\xaa\x85\x11\x71\x7d\x67\x29\x67\xdd\x30\x4a\xfc\x3b\xfb\x47\xa7\xc6\xe4\x0b\x51\xe6\x7f\xd0\xe4\x93\x5f\xcf\xe4\xe7\x2f\x9e\x1f\xfc\x76\xf8\x6b\xda\x7d\xf2\x0b\x2e\xf1\xa3\xc3\x57\x47\xaf\x8e\x7f\x3b\x7c\xf5\xf2\xbf\xcb\xf6\xe2\xec\x3d\xfc\x59\xa1\xb6\x1f\x8d\x7b\x31\x27\xa3\xc7\x7b\xb3\x38\x3e\x52\xd0\x9b\xf9\x29\xfe\xdf\xd9\x02\xee\x35\xdb\xf6\x3a\x1f\xf9\xaf\x64\x16\x99\xd1\x6b\xfe\xc6\x03\x5f\x38\x2b\x0d\xf9\xc8\x23\xe6\x50\xab\x16\xe7\x59\x82\xa8\xf0\x91\xfd\x94\x3e\x4e\x75\xe5\xf5\x17\xb8\xef\xf6\x98\x98\xb3\xfe\x91\xe2\x91\xad\xd0\xe9\x25\x64\x32\xbd\x5f\x7f\xcb\xfc\x4e\x2b\xb4\xad\x64\xbd\x62\xaa\x0a\x4f\x6e\xc5\x50\x3e\x39\x7e\x88\x45\xf8\xe2\xd5\x01\x73\x42\x32\xad\xa5\x65\xf4\xf1\x26\xaf\xc9\x32\xa4\xae\xca\xb7\x26\xa6\x62\x4c\x1a\x0c\xfe\x91\xdd\x11\x15\x18\x3c\x10\xc1\xfd\x43\x7c\x60\x1d\x18\x76\xb3\x62\x73\x6e\x2a\x05\x8c\xff\xe4\xc9\x15\xdc\x51\x5b\x4e\x67\x1f\xd9\x9b\xd9\x29\xbb\x3c\x9b\xbd\x61\x57\x67\xb3\xcb\xd3\xb7\x0f\x31\xe6\xc6\xd5\x43\x20\x51\xd6\x8c\xbf\x13\xd2\x0f\x03\xca\xda\x45\xeb\x58\x25\xac\x5d\xc8\xf4\x00\x6b\xdb\x14\x8d\x62\xf7\x2c\x58\xe2\x9c\x14\x94\x2a\xab\xb9\x34\x34\xb3\x47\x08\x73\x8b\x21\x4b\xf6\xb3\xee\x7a\x74\x98\x72\x7f\x9c\x73\x0f\x8f\xee\xf0\xab\x3f\xaf\xd8\xec\xcd\xf9\xbb\x0f\x0f\x71\x75\xef\xdf\x9f\x75\x7a\x1f\x1e\x9c\x1c\xbc\x3c\x78\x79\xf2\xf2\x78\x87\xd3\xfb\x7c\x76\xf5\xf7\xb3\x37\x3f\xe6\x10\xff\x66\xe7\x3d\xfb\xae\x5d\xfa\x78\xae\x7d\xfe\x88\xae\x8d\x66\xee\x77\x2f\x8c\x10\xc0\xef\x5b\x5f\x81\x97\xa6\xd9\xaf\xad\xdf\xef\x8c\xda\x97\xb8\x5f\xc9\xba\x06\x0f\x9b\x5d\xf1\x33\x22\xde\x6e\x2b\xff\xab\x73\xe1\xe1\xc6\xda\xc7\x8f\x89\x51\xd1\xff\xbb\xb8\xb8\x7e\xd0\x0e\x9d\xfb\xa7\x52\xfd\xf4\xed\xe5\x03\x95\xef\x5f\x88\x98\xb6\x55\x9b\x9e\x83\x63\xf4\x2e\xa8\x7a\x0b\x77\xdb\x82\x5f\x6d\xb8\xa7\x30\xff\xfc\xe2\xcd\x3f\xdf\x9f\x3d\x28\x63\x14\xe2\x69\x66\xee\xe3\xec\xf4\xf4\x41\x13\xb7\x10\xe9\x1b\x81\xf3\x56\x00\x22\x5b\x6a\x56\x7c\x85\x1b\xf8\xe2\x7b\x5a\xf0\x5c\x3c\xcd\x9c\x7d\xbc\xbe\x9c\x9d\x3e\x68\xce\xa4\x75\xe4\x97\x02\x0e\xbc\x7e\x12\x43\x2e\x67\xbf\xbf\xbb\x78\x50\x56\xbc\xe5\xf7\x19\x38\xfe\x6d\xdc\xe6\xf9\xf4\x29\x6c\xbc\x7e\x77\xfe\xa0\xa9\x5a\xce\xb9\x69\x5a\xf7\x34\xba\x5f\xff\xc9\x4e\x2f\x3e\xfc\xff\xbb\xbf\x7d\x83\x05\x7b\x93\x4f\x7b\x5f\xf6\xf6\xfe\x13\x00\x00\xff\xff\xf1\x77\x31\xfd\x05\x2d\x00\x00") |
There was a problem hiding this comment.
I'm going to add details in #97
fabiokung
reviewed
May 2, 2018
| @@ -63,7 +61,8 @@ type GPUContainer interface { | |||
|
|
|||
| // Container contains config state for a container. | |||
| // It is not safe to be used concurrently, synchronization and locking needs to be handled externally. | |||
| type Container struct { // nolint: maligned | |||
| type Container struct { | |||
| // nolint: maligned | |||
There was a problem hiding this comment.
gofmt has been doing this for me too (breaking the nolint to a different line). Does this break the linter?
There was a problem hiding this comment.
It just means you have to write no lint in every "paragraph" -- It might be new behaviour on go 1.10?
fabiokung
reviewed
May 2, 2018
| @@ -1,4 +1,4 @@ | |||
| FROM ubuntu:xenial | |||
| FROM ubuntu:artful | |||
There was a problem hiding this comment.
maybe consider going straight to bionic?
There was a problem hiding this comment.
Docker is not yet support on bionic. :(.
fabiokung
reviewed
May 2, 2018
hack/make/lint.mk
Outdated
| @@ -19,5 +19,5 @@ else | |||
| CHECKSTYLE := | |||
| endif | |||
|
|
|||
| GOMETALINTER := gometalinter --vendor --tests --vendored-linters $(CHECKSTYLE) --disable=gotype --enable=unused --enable=goimports --enable=gofmt --concurrency=$(NPROCS) --deadline=600s --exclude=api/netflix/titus --exclude=/usr/local/go/src | |||
| GOMETALINTER := gometalinter --vendor --tests --vendored-linters $(CHECKSTYLE) --disable=gotype --enable=unused --enable=goimports --enable=gofmt --concurrency=$(NPROCS) --deadline=600s --exclude=executor/runtime/docker/seccomp --exclude=api/netflix/titus --exclude=/usr/local/go/src | |||
There was a problem hiding this comment.
maybe move the non-lintable pieces to a separate file and exclude it, instead of the whole package?
There was a problem hiding this comment.
The (entire) package is generated. Let me add some stuff about it there.
There was a problem hiding this comment.
Mixing generated and non-generated code in a single package seems to end poorly. Mostly because the generator internally relies on some private methods, and variables that are "unstable" API-wise. Mixing code in the same package opens you up to accidentally foot shooting.
There was a problem hiding this comment.
OK, I didn't know the entire package was generated. If that is the case, 👍 to ignoring it all.
fabiokung
reviewed
May 2, 2018
| interfaces: 3, | ||
| ipAddressesPerInterface: 10, | ||
| ip6AddressesPerInterface: 10, | ||
| // Maybe? |
There was a problem hiding this comment.
@aather Can you confirm the network throughput numbers here? Assuming best case?
|
@sargun looking good so far. I can't comment much on the correctness of the seccomp and apparmor profiles. I'd say carry on for now, but what are your thoughts on auditing/ensuring it has a good enough protection moving forward? |
|
On a more serious note -- a lot of the code here that's "dangerous" is dark code (behind nested containers), which has to be set by the master. What do you guys think of doing a real code review so, we can get this into Titus standalone, and folks testing? |
|
Sorry, I can take a more serious look soon. Is there anyone (MCE, maybe) waiting on it and/or would use it if it were merged sooner? |
|
Sorry, maybe more context would have been better. There's a lot changing here, and a lot of moving parts. It means that whenever we change anything in the executor, this PR requires a lot of refactoring. There are other people who I'd like to give this PR to, so they can start testing it, specifically:
Given that a lot of this code is "dark" code, I think that given all of these things which are "coming soon" (tm), would be better to enable now, so we can find where the issues in our dark code lies. If it would be easier, I can split the code into the following parts:
|
|
@sargun merging as-is behind the feature flag seems reasonable to me. To me the main thing is still getting the protections around |
|
Agreed. Don't wait on me for merging the PR. I'll take a look when I can, but will catch up next week either way. |
sargun
force-pushed
the
nested-containers-2
branch
from
c6babf7
to
8d9bfc2
Compare
Pull Request Test Coverage Report for Build 484
💛 - Coveralls |
This bumps the builders to artful, so they have access to the right headers in order to get access to cgroup unshare. It also bumps tini to pull in this version: Netflix-Skunkworks/tini#2 There are a few things that have changed: - A new seccomp profile for nested containers - A new apparmor profile for nested containers This PR still has a few TODOs: Lock down the following syscalls: * Clone: No making new namespaces * Mount: Only allow tmpfs * Unshare: Only allow cgroups Figure out how to better secure systemd * Custom LSM that doesn't allow it to give away the machine
sargun
force-pushed
the
nested-containers-2
branch
from
8d9bfc2
to
7c1d235
Compare
Codecov Report
@@ Coverage Diff @@
## master #91 +/- ##
===========================================
+ Coverage 24.29% 35.08% +10.78%
===========================================
Files 64 65 +1
Lines 7165 7316 +151
===========================================
+ Hits 1741 2567 +826
+ Misses 5223 4439 -784
- Partials 201 310 +109
|
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This bumps the builders to artful, so they have access to the
right headers in order to get access to cgroup unshare.
It also bumps tini to pull in this version:
Netflix-Skunkworks/tini#2
There are a few things that have changed:
This PR still has a few TODOs:
Lock down the following syscalls:
Figure out how to better secure systemd
away the machine