Skip to content

process: implement thread-backed fork()/vfork()/execve()/waitpid() (opt-in, off by default) - #1455

Open
gburd wants to merge 2 commits into
cloudius-systems:masterfrom
gburd:wip/feat-fork
Open

process: implement thread-backed fork()/vfork()/execve()/waitpid() (opt-in, off by default)#1455
gburd wants to merge 2 commits into
cloudius-systems:masterfrom
gburd:wip/feat-fork

Conversation

@gburd

@gburd gburd commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

What

Adds a thread-backed fork()/vfork()/execve()/waitpid() implementation to OSv, gated entirely behind a new CONFIG_fork kconfig option that defaults to OFF. With the flag off (the default), none of this code is compiled and OSv is byte-for-byte the current single-address-space kernel — fork()/vfork() return ENOSYS exactly as today. The feature is opt-in for workloads that need fork().

Why

OSv is a single-address-space unikernel, so a literal copy-on-write fork() is not its model, and fork() has historically been a stub. But a large class of real Linux programs use fork() — most commonly fork()+exec() to spawn a child program, plus system()/popen(). This provides the useful, compatible subset without disturbing the default OSv model.

How

With CONFIG_fork enabled:

  • fork()/vfork() create a child OSv thread that resumes in fork()'s caller and returns 0 in the child / the child pid in the parent (the classic twin return). The child runs on a private copy of the parent's user stack (so parent and child have independent locals after the return) and gets its own fresh OSv per-thread TLS block (own errno, etc.). Arch halves in arch/x64/fork.cc and arch/aarch64/fork.cc.
  • execve() launches the target as a fresh OSv application (its own ELF namespace) and does not return — making fork()+exec() work.
  • waitpid()/wait4()/wait() reap a child's exit status via a pid→child registry; SIGCHLD is raised to the parent on child exit (and SIGCHLD/SIGURG/SIGWINCH now correctly default to ignore rather than powering off the VM).
  • exit()/_exit() in a fork child ends only that child, not the whole unikernel.
  • pthread_atfork prepare/parent/child handlers are now actually run around fork() (they were a no-op stub; glibc/musl register these internally, e.g. to reset the malloc arena lock in the child).
  • sys_clone() routes the non-CLONE_THREAD (fork) case here when enabled.

Validation

tst-fork (new) passes 10/10 on both x86-64 and aarch64: twin return, private-stack isolation (parent's local intact after the child mutates its own copy), fork()+exec(), vfork(), and waitpid() reaping exit codes. Built and confirmed in both configurations: CONFIG_fork off builds with the fork code fully excluded and the kernel healthy; CONFIG_fork on builds and passes tst-fork.

Scope / honest limitations (documented in documentation/fork.md)

This base does not give the child private memory: it shares the parent's heap and globals (only the stack is copied). That is fine for fork()+exec(), system(), and children that only read shared state before exec/exit. A follow-up adds per-child copy-on-write address spaces (behind the same flag) for programs that need memory-isolated multi-process fork(). Also documented as not carried by this base: deep-call-chain child unwind, fork()-as-memory-snapshot (e.g. Redis BGSAVE), and a separate pre-existing fault in execve()'s new-ELF-namespace path.

Off by default, so there is no risk to existing OSv builds; this simply makes fork() available to those who opt in.

@gburd

gburd commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Added a fix for a fork-child lifecycle leak found while validating execve(): a forked child was created as an attached thread, so it held a shared_ptr to the app's application_runtime that nothing ever released (the parent reaps the child via waitpid/the pid registry, never join()). The runtime refcount therefore never hit zero, ~application_runtime never fired, and the loader's application::join() blocked forever -> OSv hung at shutdown after a successful fork+exec. Fix: create the fork child detached and dispose() it after its exit bookkeeping so the reaper releases application_runtime and OSv shuts down cleanly. Also confirmed execve() itself works (launches the target program via application::run in a new ELF namespace) and re-enabled tst-fork's fork+exec test to exec a real payload; added tst-execve. Validated on x86-64/KVM: tst-execve 3/3, tst-fork 10/10, 5x repeat clean, clean shutdown, no non-fork regressions.

gburd added a commit to gburd/osv-1 that referenced this pull request Jul 24, 2026
…d inherited (backend connection-socket wall)

OSv has ONE GLOBAL fd table (fs/vfs/kern_descrip.cc gfdt[]) shared by a
fork parent and child.  0a87423 handled a fork CHILD closing an fd it
inherited (drop only the childs ref, keep the shared slot for the
parent).  It did NOT handle the reverse: the top-level OWNER (a
postmaster) closing a connection fd that a LIVE forked child still needs.

PostgreSQLs postmaster->backend handoff hit exactly this:
  AcceptConnection() -> gfdt[N] = accepted socket (f_count=1)
  BackendStartup() -> fork() (snapshot fholds N -> f_count=2; child inherits N)
  postmaster: closesocket(s.sock) == close(N)   // no longer needed here
The postmaster is NOT a fork child, so fork_child_close_inherited_fd()
returned false and the normal fdclose() nulled the SHARED gfdt[N] slot.
The forked backend then either saw gfdt[N]==NULL -> getsockname()=EBADF,
or the freed fd was reused by the backends own startup open() for a
regular file (DTYPE_VNODE) -> getsockname()=ENOTSOCK.  Stock PG18.4 died
with FATAL: getsockname() failed: ENOTSOCK/EBADF (pqcomm.c pq_init) and
the psql connection was closed before any query was served.

gdb (KVM+hbreak on kern_getsockname) proved it: at the failing
getsockname(fd=31) the shared slot had been closed by the postmaster and
re-used -- gfdt[31] pointed at a file with f_type=1 (DTYPE_VNODE), not a
socket, so getsock_cap()s file_type()!=DTYPE_SOCKET returned ENOTSOCK.

Fix (all #if CONF_fork; non-fork path byte-identical):
 * fork_owner_close_inherited_fd(fd, fp): when a NON-child context closes
   an fd that a live child inherits (same fd->file), fdclose() keeps the
   shared gfdt slot (only the owners reference is fdrop()d) and records
   the fd in g_owner_released_fds -- the owner has relinquished the slot.
 * The LAST inheriting child to close/teardown such an owner-released fd
   clears the slot (fork_clear_gfdt_slot_if), so the fd can be reused.
   A slot the owner still holds is never cleared by a child (restores the
   0a87423 invariant, keeping tst-fork-socket green).
 * gfdt_lock is never nested inside g_fd_lock: the clear is deferred until
   after g_fd_lock is released.

Regression test tests/tst-fork-conn-socket.cc mirrors the handoff:
reproduces on HEAD (getsockname errno=EBADF/ENOTSOCK), passes with the fix.

Result: stock PG18.4 forked backend now passes getsockname (0 getsockname
errors across the run) and advances past the socket handoff.  The first
query is still blocked by a SEPARATE POSIX-shm/DSM gap (shm_open
/PostgreSQL.<n> ENOENT) and a net-RX fault -- distinct next walls.

Relates to shipping PR cloudius-systems#1455 (fork fd/socket inheritance).

Author: Greg Burd
gburd added a commit to gburd/osv-1 that referenced this pull request Jul 31, 2026
…d inherited (backend connection-socket wall)

OSv has ONE GLOBAL fd table (fs/vfs/kern_descrip.cc gfdt[]) shared by a
fork parent and child.  0a87423 handled a fork CHILD closing an fd it
inherited (drop only the childs ref, keep the shared slot for the
parent).  It did NOT handle the reverse: the top-level OWNER (a
postmaster) closing a connection fd that a LIVE forked child still needs.

PostgreSQLs postmaster->backend handoff hit exactly this:
  AcceptConnection() -> gfdt[N] = accepted socket (f_count=1)
  BackendStartup() -> fork() (snapshot fholds N -> f_count=2; child inherits N)
  postmaster: closesocket(s.sock) == close(N)   // no longer needed here
The postmaster is NOT a fork child, so fork_child_close_inherited_fd()
returned false and the normal fdclose() nulled the SHARED gfdt[N] slot.
The forked backend then either saw gfdt[N]==NULL -> getsockname()=EBADF,
or the freed fd was reused by the backends own startup open() for a
regular file (DTYPE_VNODE) -> getsockname()=ENOTSOCK.  Stock PG18.4 died
with FATAL: getsockname() failed: ENOTSOCK/EBADF (pqcomm.c pq_init) and
the psql connection was closed before any query was served.

gdb (KVM+hbreak on kern_getsockname) proved it: at the failing
getsockname(fd=31) the shared slot had been closed by the postmaster and
re-used -- gfdt[31] pointed at a file with f_type=1 (DTYPE_VNODE), not a
socket, so getsock_cap()s file_type()!=DTYPE_SOCKET returned ENOTSOCK.

Fix (all #if CONF_fork; non-fork path byte-identical):
 * fork_owner_close_inherited_fd(fd, fp): when a NON-child context closes
   an fd that a live child inherits (same fd->file), fdclose() keeps the
   shared gfdt slot (only the owners reference is fdrop()d) and records
   the fd in g_owner_released_fds -- the owner has relinquished the slot.
 * The LAST inheriting child to close/teardown such an owner-released fd
   clears the slot (fork_clear_gfdt_slot_if), so the fd can be reused.
   A slot the owner still holds is never cleared by a child (restores the
   0a87423 invariant, keeping tst-fork-socket green).
 * gfdt_lock is never nested inside g_fd_lock: the clear is deferred until
   after g_fd_lock is released.

Regression test tests/tst-fork-conn-socket.cc mirrors the handoff:
reproduces on HEAD (getsockname errno=EBADF/ENOTSOCK), passes with the fix.

Result: stock PG18.4 forked backend now passes getsockname (0 getsockname
errors across the run) and advances past the socket handoff.  The first
query is still blocked by a SEPARATE POSIX-shm/DSM gap (shm_open
/PostgreSQL.<n> ENOENT) and a net-RX fault -- distinct next walls.

Relates to shipping PR cloudius-systems#1455 (fork fd/socket inheritance).

Author: Greg Burd
gburd added 2 commits July 31, 2026 09:15
…pt-in)

OSv is a single-address-space unikernel, so a literal copy-on-write fork() is
not the model.  This adds a thread-backed fork() emulation that covers the
useful, compatible subset of fork semantics, gated entirely behind a new
CONFIG_fork kconfig option that defaults to OFF - so a default OSv build is
unchanged (none of this code is compiled in and fork()/vfork() return ENOSYS as
before).

With CONFIG_fork enabled:
- fork()/vfork() create a child OSv thread that resumes in fork()'s caller and
  returns 0 in the child / the child pid in the parent (the classic twin
  return).  The child runs on a private copy of the parent's user stack, so
  parent and child have independent locals after the return; it gets its own
  fresh OSv per-thread TLS block (own errno etc).
  Implemented in libc/process/fork.cc + arch/{x64,aarch64}/fork.cc.
- execve() launches the target as a fresh OSv application (its own ELF
  namespace) and does not return, making fork()+exec() work.
- waitpid()/wait4()/wait() reap a child's exit status via a pid->child
  registry; SIGCHLD is raised to the parent on child exit; SIGCHLD/SIGURG/
  SIGWINCH now correctly default to ignore (not poweroff).
- exit()/_exit() in a fork child ends only that child, not the whole unikernel.
- pthread_atfork prepare/parent/child handlers are now actually run around
  fork() (they were a no-op stub) - glibc/musl register these internally.
- sys_clone() routes the non-CLONE_THREAD (fork) case here when enabled.

Validated: tst-fork passes 10/10 on both x86-64 and aarch64 (twin return,
private-stack isolation, fork+exec, vfork, waitpid reaping).

Documented limitations (documentation/fork.md): the child shares the parent's
heap/globals (no per-process memory isolation in one address space - a follow-up
adds per-child copy-on-write address spaces behind the same flag); deep-call-
chain child unwind and fork-as-memory-snapshot (Redis BGSAVE) are not carried by
this base; execve()'s new-ELF-namespace path has a separate pre-existing fault.

Copyright (C) 2026 Greg Burd
execve() (CONF_fork) launches the target as a fresh OSv application in its own
ELF namespace via application::run(new_program=true) and, matching Linux, does
not return to the caller.  A successful exec DID launch the program (the
elf::program new-namespace construction path is fine), but after the exec'd
program finished the unikernel would hang at shutdown instead of powering off:
the loader's application::join() blocked forever on the top-level app's
_terminated flag.

Root cause is the fork child thread's lifecycle, not execve or elf::program.
fork_thread() (arch/{x64,aarch64}/fork.cc) creates the child as a normal
*attached* sched::thread.  At construction the thread captures a shared_ptr to
the current application's application_runtime (sched.cc sets _app_runtime =
app->runtime()).  Nothing ever join()s the fork child -- the parent reaps it
through the fork pid registry / waitpid(), not sched::thread::join() -- so the
thread object is never destroyed and its _app_runtime shared_ptr is never
released.  With that reference outstanding the application_runtime's use count
never reaches zero, ~application_runtime never runs, the app's _terminated is
never set, and application::join() waits forever.  (This bit every fork(), and
was most visible after fork()+execve() where the child adopts the caller app's
runtime.)

Fix: create the fork child detached and dispose it in its cleanup.
- arch/{x64,aarch64}/fork.cc: mark the child attr().detached(), so on
  completion it is handed to the thread reaper (which runs its set_cleanup()),
  rather than sitting forever waiting to be joined.
- libc/process/fork.cc: the child's cleanup now also calls
  sched::thread::dispose(child) after the existing bookkeeping (record exit
  status if it fell off the end, free the copied user stack).  Disposing the
  thread releases its _app_runtime reference, letting the owning application's
  runtime drop to zero so join() completes and OSv powers off.  This mirrors
  the default detached-thread cleanup ([this]{ dispose(this); }).

Both files are compiled only under CONF_fork, so default OSv builds are
unchanged.

Tests: tst-fork test 2 now execs a real payload (/tests/payload-exit7.so, which
prints a marker and exit(7)s) instead of the previous _exit(7) sentinel that
masked whether execve actually launched anything; a return from execve() is now
treated as a failure.  Added tst-execve.so (fork+execve launches the payload and
its exit code is reaped; missing path returns -1/ENOENT) and payload-exit7.so.

Validated on x86-64 (KVM disk boot): tst-fork 10/10 and tst-execve 3/3 pass with
the real exec payload and OSv shuts down cleanly (5/5 repeat runs, no hang).

Copyright (C) 2026 Greg Burd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant