Not possible to run unitd with sudo on macOS due to fork() safety

[ruudk]

I try to run unitd with sudo, because I want to listen to 127.0.0.1:443 (not 0.0.0.0:443). But I get the following error:

sudo /opt/homebrew/opt/unit/bin/unitd --no-daemon --statedir etc/docker/nginx-unit/  --log /dev/stdout
2023/10/08 09:06:23 [info] 28205#19594237 unit 1.31.0 started
2023/10/08 09:06:23 [info] 28206#19594245 discovery started
objc[28206]: +[NSString initialize] may have been in progress in another thread when fork() was called.
objc[28206]: +[NSString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
2023/10/08 09:06:23 [alert] 28205#19594237 process 28206 exited on signal 6

It doesn't crash though:

pstree -s unitd
-+= 00001 root /sbin/launchd
..
         \-+= 28617 root sudo /opt/homebrew/opt/unit/bin/unitd --no-daemon --statedir etc/docker/nginx-unit/ --log /dev/stdout
           \--- 28618 root unit: main v1.31.0 [/opt/homebrew/opt/unit/bin/unitd --no-daemon --statedir etc/docker/nginx-unit/ --log /dev/stdout]     

But it does not respond to any requests.

[lcrilly]

Thanks for the report. Can confirm I can reproduce here on macOS 13.6. Can also reproduce when building from source. Flagging for investigation.

[ac000]

As a workaround you can do

$ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
$ sudo --preserve-env=OBJC_DISABLE_INITIALIZE_FORK_SAFETY /opt/homebrew/opt/unit/bin/unitd --no-daemon --statedir etc/docker/nginx-unit/  --log /dev/stdout

[lcrilly]

Well, that was unexpected! Confirmed, that works for me.

@ac000 you may also have come across this article. Do you think the __DATA,__objc_fork_ok approach will work for Unit, or is this something to cover with documentation?

[ac000]

@lcrilly

Heh, yeah, I saw that page.

That would be another workaround, but wouldn't require user intervention.

I'm not sure that the underlying issue is something under our control. These errors happen when the discovery process tries to load the language modules, I've seen it with PHP and Python. It doesn't happen when Unit starts an external C program.

It sounds like it'll trip up anything that does a fork(2) but doesn't exec(2).

But the question remains why we only see this error when running as root and how is Unit set to run under macOS normally when you install via homebrew?

[lcrilly]

why we only see this error when running as root and how is Unit set to run under macOS normally when you install via homebrew?

The Homebrew install just installs the unitd binary - there's nothing special there to make it run as a service or anything. The same behaviour can be observed by compiling from source and running the binary. I'm also confused why elevated privileges would make a difference.

This discussion has some interesting reading
https://stackoverflow.com/questions/18854708/why-is-it-prohibited-to-use-fork-without-exec-in-mac

[ac000]

Running unit as root under macOS seems to have some bigger issues.

If I run unit as root with an empty config, I can't even hit it with curl, i.e

# curl --unix-socket /tmp/unit/control.unit.sock http://localhost/config

Just hangs...

EDIT: Confirmed that running unit and curl as a normal user works...

[ac000]

I did finally manage to get a coredump

  * frame #0: 0x00007ff81b65fa32 libsystem_kernel.dylib`__abort_with_payload + 10
    frame #1: 0x00007ff81b67d82e libsystem_kernel.dylib`abort_with_payload_wrapper_internal + 82
    frame #2: 0x00007ff81b67d7dc libsystem_kernel.dylib`abort_with_reason + 19
    frame #3: 0x00007ff81b3206bd libobjc.A.dylib`_objc_fatalv(unsigned long long, unsigned long long, char const*, __va_list_tag*) + 114
    frame #4: 0x00007ff81b32064b libobjc.A.dylib`_objc_fatal(char const*, ...) + 138
    frame #5: 0x00007ff81b300ce0 libobjc.A.dylib`initializeNonMetaClass + 976
    frame #6: 0x00007ff81b300961 libobjc.A.dylib`initializeNonMetaClass + 81
    frame #7: 0x00007ff81b300961 libobjc.A.dylib`initializeNonMetaClass + 81
    frame #8: 0x00007ff81b31595b libobjc.A.dylib`initializeAndMaybeRelock(objc_class*, objc_object*, locker_mixin<lockdebug::lock_mixin<objc_lock_base_t>>&, bool) + 221
    frame #9: 0x00007ff81b300665 libobjc.A.dylib`lookUpImpOrForward + 778
    frame #10: 0x00007ff81b302d4f libobjc.A.dylib`object_setClass + 118
    frame #11: 0x00007ff81b6f723b CoreFoundation`_CFRuntimeCreateInstance + 709
    frame #12: 0x00007ff81b6f6831 CoreFoundation`__CFStringCreateImmutableFunnel3 + 2145
    frame #13: 0x00007ff81b6f5fbe CoreFoundation`CFStringCreateWithCString + 67
    frame #14: 0x00007ff81c5a340c Foundation`-[NSProcessInfo arguments] + 68
    frame #15: 0x00007ff81b302183 libobjc.A.dylib`load_images + 888
    frame #16: 0x00007ff81b3466d8 dyld`dyld4::RuntimeState::notifyObjCInit(dyld4::Loader const*) + 170
    frame #17: 0x00007ff81b35084f dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 167
    frame #18: 0x00007ff81b35083d dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 149
    frame #19: 0x00007ff81b35083d dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 149
    frame #20: 0x00007ff81b35083d dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 149
    frame #21: 0x00007ff81b3534b7 dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_1::operator()() const + 169
    frame #22: 0x00007ff81b3508f1 dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 93
    frame #23: 0x00007ff81b36d3f6 dyld`dyld4::APIs::dlopen_from(char const*, int, void*) + 944
    frame #24: 0x0000000102ec57c8 unitd`nxt_discovery_start [inlined] nxt_discovery_module(task=<unavailable>, mp=0x00007feb25804110, modules=0x00007feb26009a00, name="/tmp/unit/modules/python3.unit.so") at nxt_application.c:355:10 [opt]
    frame #25: 0x0000000102ec57bb unitd`nxt_discovery_start [inlined] nxt_discovery_modules(task=<unavailable>, path=<unavailable>) at nxt_application.c:245:15 [opt]
    frame #26: 0x0000000102ec56f9 unitd`nxt_discovery_start(task=<unavailable>, data=<unavailable>) at nxt_application.c:176:9 [opt]
    frame #27: 0x0000000102e92cdb unitd`nxt_process_do_start(task=0x00007feb2500ac00, process=0x00007feb25d045a0) at nxt_process.c:740:15 [opt]
    frame #28: 0x0000000102e92586 unitd`nxt_process_start [inlined] nxt_process_setup(task=0x00007feb2500ac00, process=0x00007feb25d045a0) at nxt_process.c:701:15 [opt]
    frame #29: 0x0000000102e9257b unitd`nxt_process_start [inlined] nxt_process_create(task=0x00007feb2500ac00, process=0x00007feb25d045a0) at nxt_process.c:579:15 [opt]
    frame #30: 0x0000000102e9257b unitd`nxt_process_start(task=0x00007feb2500ac00, process=0x00007feb25d045a0) at nxt_process.c:216:11 [opt]
    frame #31: 0x0000000102e91e64 unitd`nxt_process_init_start(task=0x00007feb2500ac00, init=nxt_process_init_t @ 0x00007ff7bd070f20) at nxt_process.c:170:11 [opt]
    frame #32: 0x0000000102eaff73 unitd`nxt_main_process_start(thr=0x00007feb24f04590, task=0x00007feb2500ac00, rt=0x00007feb24f04a20) at nxt_main_process.c:103:12 [opt]
    frame #33: 0x0000000102ea8627 unitd`nxt_runtime_initial_start(task=0x00007feb2500ac00, status=<unavailable>) at nxt_runtime.c:420:9 [opt]
    frame #34: 0x0000000102ea16c2 unitd`nxt_event_engine_start(engine=0x00007feb2500ac00) at nxt_event_engine.c:542:13 [opt]
    frame #35: 0x0000000102ecc634 unitd`main.cold.1 at nxt_main.c:35:5 [opt]
    frame #36: 0x0000000102e8f8f9 unitd`main(argc=<unavailable>, argv=<unavailable>) at nxt_main.c:33:5 [opt]
    frame #37: 0x00007ff81b33a41f dyld`start + 1903

That is from simply trying to load the python language module, which happens at frame 24.

At frame 29 we do call fork(2).

Unit makes a lot of calls to fork(2) without ever calling exec(2). It's how all the application processes are created for example.

Why this issue only triggers as root will remain a mystery for now...

The best option looks to be to see if we can fixup the unit binary in macOS to include this __DATA,__objc_fork_ok section, seeing as fork() is fundamental to how Unit works...

(I have never seen any other Unix behave like this...)

[ac000]

Looks like documentation may be the answer currently...

I put the following in src/nxt_main.c

asm(".section __DATA, __objc_fork_ok\n.long 0\n");

Which gives

$ otool -s __DATA __objc_fork_ok /tmp/unit/sbin/unitd 
/tmp/unit/sbin/unitd:
Contents of (__DATA,__objc_fork_ok) section
000000010004f920    00 00 00 00

(I don't think it matters what the actual contents are...)

But problem remains...

[ghost]

Doc-wise: https://unit.nginx.org/installation/#homebrew

[ac000]

This no longer seems to be an issue on at least macOS 14.4.1