Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MacOS High Sierra Apache, Case-insensitive FS causes fork() sanitizer crash. #1986

Closed
jithinraj opened this issue Oct 8, 2017 · 96 comments
Closed

MacOS High Sierra Apache, Case-insensitive FS causes fork() sanitizer crash. #1986

jithinraj opened this issue Oct 8, 2017 · 96 comments

Comments

@jithinraj
Copy link

jithinraj commented Oct 8, 2017 •
edited by CamJN
Loading

Help please:
https://stackoverflow.com/questions/46628052/phusion-passenger-installation-error-in-macos-10-13-high-sierra

## Issue report
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: buildout/apache2/module_libboost_oxt.a(regex_debug.o) has no symbols
/Library/Developer/CommandLineTools/usr/bin/ranlib: file: buildout/apache2/module_libboost_oxt.a(tss_null.o) has no symbols
clang -o buildout/apache2/mod_passenger.o  -Isrc/agent -Isrc/cxx_supportlib -Isrc/cxx_supportlib/vendor-copy -Isrc/cxx_supportlib/vendor-modified -fPIC -D_REENTRANT -I/usr/local/include -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wpointer-arith -Wwrite-strings -Wno-long-long -Wno-missing-field-initializers -fvisibility=hidden -DVISIBILITY_ATTRIBUTE_SUPPORTED -g -DHAS_ALLOCA_H -DHAS_SFENCE -DHAS_LFENCE -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS -c src/apache2_module/mod_passenger.c
src/apache2_module/mod_passenger.c:26:10: fatal error: 'httpd.h' file not found
#include <httpd.h>
         ^~~~~~~~~
1 error generated.
rake aborted!
Command failed with status (1): [clang -o buildout/apache2/mod_passenger.o  -Isrc/agent -Isrc/cxx_supportlib -Isrc/cxx_supportlib/vendor-copy -Isrc/cxx_supportlib/vendor-modified -fPIC -D_REENTRANT -I/usr/local/include -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wpointer-arith -Wwrite-strings -Wno-long-long -Wno-missing-field-initializers -fvisibility=hidden -DVISIBILITY_ATTRIBUTE_SUPPORTED -g -DHAS_ALLOCA_H -DHAS_SFENCE -DHAS_LFENCE -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS -c src/apache2_module/mod_passenger.c]
/private/tmp/passenger-20171008-7392-owxt2w/passenger-5.1.8/build/support/cplusplus.rb:53:in `run_compiler'
/private/tmp/passenger-20171008-7392-owxt2w/passenger-5.1.8/build/support/cplusplus.rb:97:in `compile_c'
/private/tmp/passenger-20171008-7392-owxt2w/passenger-5.1.8/build/support/cplusplus.rb:154:in `block in define_c_object_compilation_task'
Tasks: TOP => apache2 => buildout/apache2/mod_passenger.so => buildout/apache2/mod_passenger.o
@OnixGH
Copy link
Contributor

OnixGH commented Oct 8, 2017 •
edited
Loading

High Sierra support was recently added to stable and will be part of the upcoming release 5.1.11.

@OnixGH OnixGH closed this as completed Oct 8, 2017
@jithinraj
Copy link
Author

Thanks for the speedy response.

@FooBarWidget FooBarWidget added this to the 5.1.11 milestone Oct 18, 2017
@patricksereno
Copy link

I don't see that it's fixed in 5.1.11. I'm running 10.13 (17A405) both with/without "Command Line Tools (macOS High Sierra version 10.13) for Xcode" (version 9.0) installed

I was able to get the compile to complete by modifying apache.rb as such:

*** apache.rb	2017-10-18 08:33:42.000000000 -0700
--- apache-fix.rb	2017-10-18 08:34:13.000000000 -0700
*************** module PhusionPassenger
*** 614,619 ****
--- 614,620 ----
          if os_name_simple == 'macosx' && os_version >= '10.13' && httpd == '/usr/sbin/httpd'
            xcode_prefix = `/usr/bin/xcode-select -p`.strip
            flags << "-I#{xcode_prefix}/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/apache2"
+           flags << "-I/usr/include/apache2"
          end
        else
          apxs2_flags = `#{apxs2} -q CFLAGS`.strip << " -I" << `#{apxs2} -q INCLUDEDIR`.strip
*************** module PhusionPassenger
*** 816,821 ****
--- 817,823 ----
              xcode_prefix = `/usr/bin/xcode-select -p`.strip
              ["-I#{xcode_prefix}/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/apr-1",
                '-lapr-1']
+             ['-I/usr/include/apr-1', '-lapr-1']
            else
              ['-I/usr/include/apr-1', '-lapr-1']
            end

If you're running home-brew ruby, here's the process

  1. Download the above code to /tmp/apache.rb.patch
  2. cd /usr/local/lib/ruby/gems/2.4.0/gems/passenger-5.1.11/src/ruby_supportlib/phusion_passenger/platform_info
  3. patch < /tmp/apache.rb.patch

@CamJN
Copy link
Member

CamJN commented Oct 18, 2017 •
edited
Loading

@patricksereno you are installing from a gem, that is a different process than homebrew, perhaps open another issue?

@patricksereno
Copy link

patricksereno commented Oct 19, 2017 via email •
edited by CamJN
Loading

@FooBarWidget
Copy link
Member

FooBarWidget commented Oct 19, 2017 •
edited
Loading

@patricksereno It looks like your High Sierra system is a bit different than the ones we tested on. Can you tell me what you see if you run these command?

find / -name httpd.h 2>/dev/null
xcode-select -p

@patricksereno
Copy link

patricksereno commented Oct 19, 2017 via email

FooBarWidget added a commit that referenced this issue Oct 19, 2017
@FooBarWidget
macOS High Sierra: add another hardcoded Apache include directory loc…
c9ba024 
…ation

Apparently on some systems the Apache header files may be in a different location
than on our test systems.

Partially closes GH-1986.
@FooBarWidget
Copy link
Member

Thanks @patricksereno. I've published a branch GH-1986-high-sierra-continued-fix. Can you check whether that one compiles correctly?

And can you also show me what sw_vers outputs?

@CamJN As for his __NSCFConstantString initialize error: the message indicates that it's an Apache worker process that's crashing. Are you able to reproduce this on your system with Apache?

@patricksereno
Copy link

patricksereno commented Oct 19, 2017 via email •
edited
Loading

@CamJN
Copy link
Member

CamJN commented Oct 19, 2017

@patricksereno Are you running Apache with the event, worker, or prefork MPM module in use?

@patricksereno
Copy link

prefork. I'm using macOS Server Version 5.4 (17S1207).

This appears to default to prefork.

httpd -D DUMP_MODULES | grep mpm
 mpm_prefork_module (static)

@CamJN
Copy link
Member

CamJN commented Oct 20, 2017

@patricksereno could you apply the following patch and recompile? There will be some clang warnings about unused flags, that's fine. I haven't been able to reproduce your issue, unfortunately so I can't verify it myself.

diff --git a/src/ruby_supportlib/phusion_passenger/platform_info/apache.rb b/src/ruby_supportlib/phusion_passenger/platform_info/apache.rb
index e546f0798..fe075c21c 100644
--- a/src/ruby_supportlib/phusion_passenger/platform_info/apache.rb
+++ b/src/ruby_supportlib/phusion_passenger/platform_info/apache.rb
@@ -593,7 +593,7 @@ def self.apache2_module_cxxflags(with_apr_flags = true)
     # The C compiler flags that are necessary to compile an Apache module.
     # Also includes APR and APU compiler flags if with_apr_flags is true.
     def self.apache2_module_c_or_cxxflags(language, with_apr_flags = true)
-      flags = [""]
+      flags = ["-framework", "Foundation"]
       if (language == :c && cc_is_sun_studio?) || (language == :cxx && cxx_is_sun_studio?)
         flags << "-KPIC"
       else

@patricksereno
Copy link

patricksereno commented Oct 31, 2017 via email

@CamJN CamJN modified the milestones: 5.1.11, 5.1.13 Nov 28, 2017
@CamJN
Copy link
Member

CamJN commented Nov 28, 2017

Well at least that narrows it down. (Also some people will be able to work around this bug that way).

@OnixGH OnixGH closed this as completed in e0ffa39 Nov 28, 2017
@OnixGH OnixGH reopened this Nov 28, 2017
@CamJN
Copy link
Member

CamJN commented Nov 28, 2017

So the current workarounds are:

  • Use a non-system apache (so it doesn't have apple's patches)
  • Set the OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES env var to the apache launchd plist (requires disabling SIP)
  • Add PassengerHighPerformance on to your virtualhost if your app can work that way
  • Use a case-sensitive filesystem (requires a fresh install of macOS)

@patricksereno
Copy link

On High Sierra (10.13) with X-Code 9.1, I was able to install the ruby gem (5.1.12), along with the patch to make it compile:

*** apache-fix.rb	2017-12-11 23:26:43.000000000 -0700
--- apache.rb	2017-12-11 23:26:21.000000000 -0700
*************** module PhusionPassenger
*** 614,619 ****
--- 614,620 ----
          if os_name_simple == 'macosx' && os_version >= '10.13' && httpd == '/usr/sbin/httpd'
            xcode_prefix = `/usr/bin/xcode-select -p`.strip
            flags << "-I#{xcode_prefix}/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/apache2"
+           flags << "-I/usr/include/apache2"
          end
        else
          apxs2_flags = `#{apxs2} -q CFLAGS`.strip << " -I" << `#{apxs2} -q INCLUDEDIR`.strip
*************** module PhusionPassenger
*** 816,821 ****
--- 817,823 ----
              xcode_prefix = `/usr/bin/xcode-select -p`.strip
              ["-I#{xcode_prefix}/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/apr-1",
            '-lapr-1']
+             ['-I/usr/include/apr-1', '-lapr-1']
            else
              ['-I/usr/include/apr-1', '-lapr-1']
            end

And then include in the VirtualHost configuration:

PassengerHighPerformance on

However, trying to access a file from public results in the following apache error message:

objc[20681]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[20681]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
[Mon Dec 11 22:36:53.335695 2017] [core:notice] [pid 20493] AH00052: child pid 20681 exit signal Abort trap (6)

I would be inclined to blame the Sinatra 2.0.0 gem, but this is the same error that was occurring for '/' before.

(Of course, this is much better than the previous state.)

@CamJN
Copy link
Member

CamJN commented Dec 14, 2017

Yeah, it's going to be a game of whack a mole until Apple fixes their end. You can help that happen faster by opening bug reports at https://bugreport.apple.com/ as they prioritize duplicated bugs.

@mattbob
Copy link

mattbob commented Jan 1, 2018

Any update on this? I tried what @NDuggan suggested using the brew version of apache but that didn't fix the issue for me. passenger-status and passenger-config validate-install all look good but I get this error over and over

objc[61064]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[61064]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
[Mon Jan 01 16:44:20.814012 2018] [core:notice] [pid 61044] AH00052: child pid 61064 exit signal Abort trap (6)

Passenger: 5.1.12 - Apache 2.4.28 - macOS: 10.13.2

@CamJN CamJN changed the title MacOS High Sierra Apache, Case-sensitive FS causes fork() sanitizer crash. MacOS High Sierra Apache, Case-insensitive FS causes fork() sanitizer crash. Jan 2, 2018
@CamJN
Copy link
Member

CamJN commented Jan 2, 2018

@mattbob if there's a crash in the homebrew apache, that would be surprising, as it doesn't have the patches that invoke the Obj-C runtime to load. Can you get a backtrace?

@enrique-ramirez
Copy link

I'm not sure if this is the same error as @mattbob .

I'm running a ruby app on an SSL virtual host using passenger. This is my config: https://gist.github.com/enrique-ramirez/6173de71b866ce2e3b6cb78a94b84fe7

I also want to be able to access localhost via http (port 80) and localhost/~userName folders.

For some reason, I cannot get both of them running. The gist config works fine for my ruby app, but when attempting to access localhost (default Apache DocumentRoot), it always says ERR_EMPTY_RESPONSE and when checking the logs, I see the exact same error @mattbob has:

objc[27567]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[27567]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
[Thu Jan 04 16:41:04.006967 2018] [core:notice] [pid 27258] AH00052: child pid 27567 exit signal Abort trap (6)

If I do not load the passenger_module, apache is back to normal but, of course, my rails app does not run.

I'm sorry if this is already answered, but there are so many comments it's a bit confusing: Is there a way to get both of them running? I develop a few PHP projects, so I need apache to run them as well.

@CamJN CamJN mentioned this issue Jan 8, 2018
Closed
@CamJN
Copy link
Member

CamJN commented Jan 8, 2018

Yes, that sounds like the same bug, please file a bug report with apple, the more they get the sooner they'll fix this.

@vinz84
Copy link

vinz84 commented Jan 16, 2018 •
edited
Loading

@enrique-ramirez deactivate apache that come with osx and reinstall like this instruction:
https://getgrav.org/blog/macos-sierra-apache-multiple-php-versions

Will work for me (passenger 5.1.12, Xcode 9.1, High Sierra 10.13.2, ruby-2.4.1 via rvm)

@enrique-ramirez
Copy link

enrique-ramirez commented Jan 17, 2018 •
edited
Loading

@vinz84 Thanks! I'll give it a go once I have a bit more time. I resorted to MAMP for the time being.

@CamJN I also filed a bug report with apple, which was marked as a duplicate after a little while. Hope it helps to resolve the issue faster!

@OnixGH OnixGH removed this from the 5.1.13 milestone Jan 23, 2018
@CamJN
Copy link
Member

CamJN commented Jan 23, 2018

Closing this issue, because the original problem was found in Apple's Apache. If you have a problem that can be shown to not involve the system Apache then you can open another issue.

@CamJN CamJN closed this as completed Jan 23, 2018
@CamJN CamJN mentioned this issue Apr 18, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@FooBarWidget @wulffeld @vinz84 @mattbob @enrique-ramirez @NDuggan @patricksereno @CamJN @jithinraj @OnixGH