OCI runtime support for nspawn #9762
Conversation
|
/cc @alban, @iaguis, @dongsupark |
man/systemd-nspawn.xml
Outdated
|
|
||
| <listitem><para>Takes the path to an OCI runtime bundle to invoke, as specified in the <ulink | ||
| url="https://github.com/opencontainers/runtime-spec/blob/master/spec.md">OCI Runtime Specification</ulink>. In | ||
| this case no <filename>.settings</filename> file is loaded, and the root directory and various settings are |
There was a problem hiding this comment.
you mean .nspawn file here ?
|
This pull request introduces 6 alerts when merging a9451ba into 5a8b164 - view on LGTM.com new alerts:
Comment posted by LGTM.com |
|
This pull request introduces 6 alerts when merging 7572186 into 5a8b164 - view on LGTM.com new alerts:
Comment posted by LGTM.com |
src/basic/json-internal.h
Outdated
| #define JSON_VALUE_NULL ((JsonValue) {}) | ||
|
|
||
| /* We use fake JsonVariant objects for some special values, in order to avoid memory allocations for them. Note that | ||
| * effectively this means that there are multiple ways to encode the some objects: via these magic values or as |
There was a problem hiding this comment.
encode the some objects -> encode some objects
src/basic/json.h
Outdated
|
|
||
| /* | ||
| In case you wonder why we have our own JSON implementation, here are a couple of reasons why this implementation has | ||
| benefits over various other implementatins: |
There was a problem hiding this comment.
I'm not going to say it's a bold move to implement a json parser :-), but if it's going to be implemented, maybe it would be helpful to start fuzzing it right away. I ran a fuzzer compiled with UBSan for about ten seconds and it crashed after receiving relatively normal input like -34444444444541444444 and "0" with
../../src/systemd/src/basic/json.c:1845:48: runtime error: signed integer overflow: 10 * -3444444444454144444 cannot be represented in type 'long'
#0 0x7fe6beefb805 in json_parse_number /work/build/../../src/systemd/src/basic/json.c:1845:48
#1 0x7fe6beefa5ca in json_tokenize /work/build/../../src/systemd/src/basic/json.c:2009:29and
../../src/systemd/src/basic/json.c:350:9: runtime error: index 1 out of bounds for type 'char [0]'
#0 0x7fab2d4c2042 in json_variant_new_stringn /work/build/../../src/systemd/src/basic/json.c:350:22
#1 0x7fab2d4c8690 in json_parse_internal /work/build/../../src/systemd/src/basic/json.c:2321:29
#2 0x7fab2d4c781a in json_parse /work/build/../../src/systemd/src/basic/json.c:2477:16
There were also a couple warnings from clang, the first of which seems legit:
../../src/systemd/src/basic/json.c:2486:32: warning: unused variable 'opened' [-Wunused-variable]
_cleanup_fclose_ FILE *opened = NULL;
^
../../src/systemd/src/basic/json.c:2949:13: warning: variable 'source' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
if (variant) {
^~~~~~~
../../src/systemd/src/basic/json.c:2955:13: note: uninitialized use occurs here
if (source && source_line > 0 && source_column > 0)
^~~~~~
../../src/systemd/src/basic/json.c:2949:9: note: remove the 'if' if its condition is always true
if (variant) {
^~~~~~~~~~~~~
../../src/systemd/src/basic/json.c:2936:27: note: initialize the variable 'source' to silence this warning
const char *source;
^
= NULL
There was a problem hiding this comment.
../../src/systemd/src/basic/json.c:1845:48: runtime error: signed integer overflow: 10 * -3444444444454144444 cannot be represented in type 'long'
This one is misleading. If you look at the sources you find an overflow check right after, that divides the result again by 10 and checks if that result is -3444444444454144444 again.
There was a problem hiding this comment.
../../src/systemd/src/basic/json.c:350:9: runtime error: index 1 out of bounds for type 'char [0]'
And this one is misleading too. If you look at the function you'll see we actually allocate the buffer long enough. The field in the structure is the last one in it, and the only reason we specify its size as [0] rather than leave it C99-style unspecified as [] is that we sometimes want arrays of the structure, in which case we don't use the remaining buffer...
There was a problem hiding this comment.
Unfortunately, false positives are inevitable, but I don't think they make fuzzers less useful. In this particular case, __attribute__((no_sanitize("signed-integer-overflow"))) and __attribute__((no_sanitize("bounds"))) could probably be used to silence UBSan.
There was a problem hiding this comment.
I figure I could rewrite the overflow check to not trigger ubsan. But for the [0] bounds check I have no idea how I could trick it to not complain. Any idea?
Quite frankly, ubsan should be smart enough to understand that [0] is special, and generally doesn't actually mean "zero sized array", but instead is similar to C99 [], i.e. more akin to "unspecified size"...
There was a problem hiding this comment.
@poettering it's fine by me, but it shouldn't be merged into master until google/oss-fuzz#1683 is merged.
There was a problem hiding this comment.
@poettering google/oss-fuzz#1683 has been merged. I pushed the commit from #9782 on top of your branch.
There was a problem hiding this comment.
Fedora CI seems to have vanished. Does anybody know how to bring it back or trigger a build manually?
There was a problem hiding this comment.
I'm not sure what's happened, but the fuzzer is gone. It'd probably be better to leave it util later. I'll reopen #9782 so as not to forget that it isn't done yet.
src/busctl/busctl.c
Outdated
|
|
||
| json_dump_with_flags(v, stdout); | ||
| if (r < 0) | ||
| return r; |
There was a problem hiding this comment.
It'd be great if you could also take a look at the LGTM alerts. At least the last two of them don't look like false positives. For example, the if block really seems to be redundant here.
|
@giuseppe FYI |
poettering
force-pushed
the
nspawn-oci
branch
2 times, most recently
from
9913a12
to
0f5b80c
Compare
|
I force pushed a new version now, addressing all raised issues. Let's see if LGTM likes it more this time. PTAL. |
poettering
force-pushed
the
nspawn-oci
branch
2 times, most recently
from
a92797e
to
470418a
Compare
|
It probably has something to do with the absence of |
evverx
added
the
ci-fails/needs-rework 🔥
poettering
removed
the
ci-fails/needs-rework 🔥
|
Neat! CI all green now! |
|
Interestingly, it seems that LGTM restarted the analysis as soon as the label was removed. |
|
I force pushed another new version now, fixing the typo @dnicolodi found. |
|
I rebased and added a fix for #11755 on top now. |
|
|
||
| r = mount_verbose(m->graceful ? LOG_DEBUG : LOG_ERR, NULL, where, NULL, MS_BIND|MS_RDONLY|MS_REMOUNT, NULL); | ||
| if (r < 0) | ||
| return m->graceful ? 0 : r; |
There was a problem hiding this comment.
I was a bit surprised that there's no unmounting performed if the operation fails halfway with m->graceful. In practice it doesn't probably matter that much, because the file is still protected by restrictive file permissions, but I wonder it wouldn't be proper to unmount it anyway.
|
Oh, I'll submit a PR with some follow-up cleanups shortly. |
The OCI changes in systemd#9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional and log about it. Fixes systemd#12539
The OCI changes in systemd#9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional (hard fail if using OCI and log only if using nspawn cmdline). Fixes systemd#12539
The OCI changes in systemd#9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional (hard fail if using OCI and log only if using nspawn cmdline). Fixes systemd#12539
The OCI changes in systemd#9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional (hard fail if using OCI and log only if using nspawn cmdline). Fixes systemd#12539
The OCI changes in #9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional (hard fail if using OCI and log only if using nspawn cmdline). Fixes #12539
The OCI changes in systemd#9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional (hard fail if using OCI and log only if using nspawn cmdline). Fixes systemd#12539
The OCI changes in systemd#9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional (hard fail if using OCI and log only if using nspawn cmdline). Fixes systemd#12539
The OCI changes in systemd#9762 broke a use case in which we use nspawn from inside a container that has dropped capabilities from the bounding set that nspawn expected to retain. In an attempt to keep OCI compliance and support our use case, I made hard failing on setting capabilities not in the bounding set optional (hard fail if using OCI and log only if using nspawn cmdline). Fixes systemd#12539
We mean the line number in the json data, not the line number in our C source code. Addresses: systemd/systemd#9762 (comment)
if we sync the legacy and unified trees before moving to the right subcgroup then ultimately the cgroup paths in the hierarchies will be out-of-sync... Hence, let's move the payload first, and sync then. Addresses: systemd/systemd#9762 (comment) (cherry picked from commit 27da7ef) Resolves: #1837094
if we sync the legacy and unified trees before moving to the right subcgroup then ultimately the cgroup paths in the hierarchies will be out-of-sync... Hence, let's move the payload first, and sync then. Addresses: systemd/systemd#9762 (comment) (cherry picked from commit 27da7ef) Resolves: #1837094
if we sync the legacy and unified trees before moving to the right subcgroup then ultimately the cgroup paths in the hierarchies will be out-of-sync... Hence, let's move the payload first, and sync then. Addresses: systemd/systemd#9762 (comment) (cherry picked from commit 27da7ef) (cherry picked from commit 8ee1465) Resolves: #1837423
if we sync the legacy and unified trees before moving to the right subcgroup then ultimately the cgroup paths in the hierarchies will be out-of-sync... Hence, let's move the payload first, and sync then. Addresses: systemd/systemd#9762 (comment) (cherry picked from commit 27da7ef) (cherry picked from commit 8ee1465) Resolves: #1837423
This is a big one, unfortunately.
For full drop-in OCI runtime support two bits are still missing: the OCI hooks need to be executed at the right times, and we need a command line tool that provides runc compatibility. I am on both, but this already grew large enough I didn't want to pile on.
But even without those two bits it's pretty comprehensive and should pretty much work.