execve(2) support

[regularfry]

Implementing basic execve(2) support would allow simpler init process management than specifying the command line in build.mk, and make it easier to replace the java.so init process with other, custom init binaries. Is this planned?

[nyh]

Hi, to run a different command line you don't need to modify build.mk, you can specify a different command line with a "-e" option to scripts/run.py. As an example, https://github.com/cloudius-systems/osv/wiki/Running-compiled-code-on-OSv explains how build a shared object memcached.so, and then gives the example of running it as

scripts/run.py -e "memcached.so -u root -t 1"

I don't understand how execve() would help. The exec() API usually replaces the entire process with something else - and since in the OSv case the process includes both the OSv kernel and the application, replacing it all makes little sense. If you just want to load and run another shared object (without replacing OSv), you can, just use dlopen() and friends to load a DSO and run a function from it. You can see an example in elf-loader.cc, which loads a shared object and runs its main() function.

I'm closing this issue. If you still think execve() is needed, can you please reopen the issue with a better explanation why? Thanks.

[regularfry]

I was trying to separate the build process out into stages: a "kernel" core, and two filesystem overlays to add the java and test files back in. I ultimately wanted to replace the java overlay with my own, supplying my own init process (a GHC-compiled static binary was the goal, the details don't matter for these purposes), but I think an overlaid build process like this could be more generally useful.

The idea would be that the "kernel" core could be built with a static command line which just called /sbin/init. An overlay could then supply that file, which would be able to exec the overlay-specific binary with the command-line specific to that overlay. So, in the java overlay's case, the /sbin/init binary would look something like this (from memory, pardon the precise details):

#include <unistd.h>

int main(){
  char launcher[] = "/java.so";
  char *argv[] = {"-jar", "/usr/mgmt/web-1.0.jar", "app", "prod", 0};

  execve(launcher, argv, NULL);
  return 0; //keep the compiler happy
}

Given the example in elf-loader.cc, it's probably possible to implement this with dlopen(), but I'm not familiar enough with how the stack should be managed in this case to know if it's quite that simple.

Apart from a more tractable build process, the win is that no specific overlay's command-line is "blessed", so given an image, no matter how it's put together, you can always just run it without having to know the right options to run.py.

Maybe I'm being dense - I don't see how I can reopen this issue.

[nyh]

On Thu, Sep 26, 2013 at 1:10 PM, Alex Young notifications@github.comwrote:

I was trying to separate the build process out into stages: a "kernel"
core, and two filesystem overlays to add the java and test files back in. I
ultimately wanted to replace the java overlay with my own, supplying my own
init process (a GHC-compiled static binary was the goal, the details don't
matter for these purposes), but I think an overlaid build process like this
could be more generally useful.

The idea would be that the "kernel" core could be built with a static
command line which just called /sbin/init. An overlay could then supply
that file, which would be able to exec the overlay-specific binary with the
command-line specific to that overlay. So, in the java overlay's case, the
/sbin/init binary would look something like this (from memory, pardon the
precise details):

#include <unistd.h>

int main(){
char launcher[] = "/java.so";
char *argv[] = {"-jar", "/usr/mgmt/web-1.0.jar", "app", "prod", 0};

execve(launcher, argv, NULL);

This is absolutely correct, but it's just that "execve" is not the right
function given that our executable is OSv, so an execve would also replace
it.

Given the example in elf-loader.cc, it's probably possible to implement

this with dlopen(), but I'm not familiar enough with how the stack should
be managed in this case to know if it's quite that simple.

You can achieve what you wanted by more-or-less copying the code of
elf-loader.cc's run_elf (note that elf-loader.cc uses some OSv-specific API
that is an alternative to dlopen() and friends).

Would it help if I made that "run_elf" a public API in the osv:: namespace,
say osv::run?
You could then run your main as:
int main() {

int main(){ char launcher[] = "/java.so";
char *argv[] = {launcher, "-jar", "/usr/mgmt/web-1.0.jar", "app", "prod", 0};

osv::run(launcher, argv);

I think this would be a good idea.

Apart from a more tractable build process, the win is that no specific
overlay's command-line is "blessed", so given an image, no

matter how it's put together, you can always just run it without having to
know the right options to run.py

Right, this is a good idea - it just doesn't need execve ;-)

Nadav Har'El
nyh@cloudius-systems.com

[regularfry]

Right, this is a good idea - it just doesn't need execve ;-)

Heh, yes - I only want to fool the init binary just enough to get the command-line indirection to work. What I ideally have in mind is something that would provide the execve symbol for client binary linking purposes, but would replace the stack frame where it was called rather than replacing the whole OSv process. osv::run would certainly be a good starting point. It might be possible to perform some linker magic later to simply alias one to the other if symbol compatibility was important.

[nyh]

On Thu, Sep 26, 2013 at 4:40 PM, Alex Young notifications@github.comwrote:

Right, this is a good idea - it just doesn't need execve ;-)

Heh, yes - I only want to fool the init binary just enough to get the
command-line indirection to work. What I ideally have in mind is something
that would provide the execve symbol for client binary linking purposes,
but would replace the stack frame where it was called rather than replacing
the whole OSv process.

Why is it important to replace the stack frame? What's wrong that we'll
always see init's main() calling osv::run calling some-other-object.so's
main() in the main thread's stack?

I guess that we can write code to move back in the stack, and then we can
even unload the tiny init.so from memory, but I wonder what's the point.

osv::run would certainly be a good starting point. It might be possible
to perform some linker magic later to simply alias one to the other if
symbol compatibility was important.

I don't understand the compatibility issue. Do you want the same 5-line
init program to work in both Linux and OSv?

Nadav Har'El
nyh@cloudius-systems.com

[regularfry]

Just trying to get as close to POSIX as the environment allows. The more of the API that works as expected, the less surprising it is for developers down the line, and the more source that can just be dropped in without having to be modified.

However, I've just had a bit of a thought and spotted a potential problem with actually my trying to mimic execve(), and that's threads. If someone (for whatever reason) calls execve() from a background thread, they'll expect all the threads launched by that binary before the execve() call to go away. That wouldn't happen with my simplistic version, so if it's not possible to track threads such that they'd be easy to kill off, it's probably better not to fake the symbol so it's clear that there's a real semantic difference.