OS X 10.13.3 exposes bad fork() issue #155
Comments
|
Thank you for reporting this! I'll see if I can reproduce the problem. |
|
Specifics: I've recreated the behavior under the following rubies (both installed via ruby-build): If there's any other information that might help, please let me know. |
|
I tried here and it seems to work for me. Perhaps some other setting is triggering this? I see: That's with |
|
Also with 13.3, of course: |
|
OK. I'll keep digging around and see what else it might be. Thanks for looking into it! |
|
After a full day of investigations, rebuilds, reboots, and tests, none of which fixed the issue, it seems to have gone away on its own. Sorry for the wild goose chase. |
|
How frustrating! I'm glad you resolved it anyway. |
|
It looks like some people are still being bitten by this, eg.: #314 (comment) I've not been able to reproduce this on my mac, so I can't really help. If anyone can make a simple test case that shows the problem, I'll try to investigate. |
|
Hi everyone, I encounter some problem when I use ruby-vips with puma multi-workers (no problem with non-forked puma) Those lines appears in my console when I load a page from a Rails controller/view and a concurrent variant processing (active storage default setup with vips) objc[34825]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[34825]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.Use of env var My stack :
|
|
My only insight her is that libvips does not use any Objective C directly, and does not fork, so I'm not certain this error is down to libvips. Perhaps one of the (many) libvips dependencies (perhaps fontconfig?) is bringing in the objc runtime indirectly? The fork is presumably puma starting up the worker pool. It'll be running If that's the case, sleeping for a second or so after But this all sounds unlikely, and it's only a guess. |
|
I found this interesting article on the error: tldr: It sounds like you might be trying to do some processing with ruby-vips before the puma fork. The fix is ... don't! libvips won't make any threads on require. You need to not make any ruby-vips calls until the worker process has started. |
|
I have seen this error on MacOS when creating text images/labels to be applied to other images. I sidestepped it by creating a docker image to test the annotations, knowing that the production system is not a mac. I can probably create a test case... I read the article @jcupitt referred to at the time. This is a typical dragonfly job object: which sets up a sequence of actions:
It's during step 3 that the error occurs. libvips has already been in use for step 2. Caption itself:
Caveat: that is my mental model of how it works - but I've also followed it and debugged parts of caption and thumb |
|
I'm seeing this in a Rails 6.1 application that's using Where is the crash coming from?It looks like when simply loading libvips, before we even have a chance to call CRubyWhere is
|
|
Hi @shepmaster , That C stacktrace is interesting: So the I think I've found the line of code: https://github.com/ffi/ffi/blob/master/ext/ffi_c/DynamicLibrary.c#L119 I would guess that's being called from here: https://github.com/libvips/ruby-vips/blob/master/lib/vips.rb#L45 There are three lines like that in Maybe it's possible to add some We can't do much about the first two, but if it's libvips, it's probably one of the optional packages that libvips depends on, and we could maybe remove it on macos. By far the largest optional dependency for libvips is imagemagick. I'd be tempted to try building libvips without that, it could be the cause. |
Yes, I agree.
Via printf-debugging, it appears to be: Line 573 in 3206143 This makes some sense — as you mentioned:
Since the stack trace mentions libgio is used in many many places, as I'm sure you know, but a few:
Hacky Ruby Dependency Script$visited = {}
def step(fname)
output = `otool -L #{fname}`
libraries = output.lines.map { |line| line.chomp.split(' ').first }.reject { |lib| lib.include? ':' }.sort.uniq
libraries.each do |l|
unless $visited.key? l
$visited[l] = nil
$visited[l] = step(l)
end
end
end
def findem(lib)
$visited.each do |k, v|
if v.any? { |v| v.include? lib }
puts k
unless k == lib
findem(k)
end
end
end
end
step(ARGV[0])
findem('AppKit') |
|
Ahh good detective work. I found this: https://gitlab.gnome.org/GNOME/glib/-/blob/main/gio/meson.build#L384-392 So it seems that GIO pulls in appkit so it can use the macos settings framework, and I expect some other things too. I'm not sure what the best solution is. At the moment, ruby-vips does some basic start up on Maybe the best thing to do would be a very, very quick and dirty hack to delay startup, just to see if this does indeed fix the problem. Any volunteers? |
|
I have a "fix" for our application: directly adding My working theory is:
When we started out, we only added By explicitly adding This may be good enough for my team to move forward for now, but it certainly feels brittle. We have to hope that the forked process on the other end of the pipe never attempts to use
Amusingly in my case, that start up needs to be executed before the first use, before the |
|
Oh what a mess. So perhaps @janko can I ping you on this issue (sorry)? Do you have an opinion? |
|
The gem "image_processing", require: "image_processing/vips"I think it's unlikely that a Rails application will fork itself on boot, though it might when using Spring. It does seem like a safer default. |
|
Thanks @janko! That sounds like a reasonable solution. Can anyone test this? |
|
Hmmm I just checked, and I have had for some time, and I am still getting the error. However, I think that the processing I am doing (overlaying captions) isn't supplied by I am on MacOS 11.6.5 (20G527) (Big Sur), vips-8.12.2 |
|
I submitted a PR to ffi to support |
|
if RUBY_PLATFORM =~ /darwin/
normal_homebrew = (RUBY_PLATFORM =~ /\Aarm64-/ ? '/opt/homebrew' : '/usr/local')
if `brew --prefix`.chop != normal_homebrew
libs = `brew --prefix glib vips`.each_line.map { _1.chop + '/lib' }.join(':')
if ENV.has_key?('DYLD_FALLBACK_LIBRARY_PATH')
ENV['DYLD_FALLBACK_LIBRARY_PATH'] += ":#{libs}"
else
ENV['DYLD_FALLBACK_LIBRARY_PATH'] = libs
end
gem 'ffi', github: 'steakknife/ffi', branch: 'fix-155-backport', submodules: true
end
end
# The following wasn't needed
# gem "image_processing", "~> 1.2", require: "image_processing/vips"Homebrew can be used without |
This (mostly) fixes the problem reported in bullet-train-co/bullet_train-core#464 However, due to bullet-train-co/bullet_train-core#498 we don't currently try to show a user profile photo if we're using ActiveStorage instead of Cloudinary. Here's some context about why we need to add this directly to the `Gemfile`. libvips/ruby-vips#155 (comment)
* Add `ruby-vips` in the `:development, :test` group This (mostly) fixes the problem reported in bullet-train-co/bullet_train-core#464 However, due to bullet-train-co/bullet_train-core#498 we don't currently try to show a user profile photo if we're using ActiveStorage instead of Cloudinary. Here's some context about why we need to add this directly to the `Gemfile`. libvips/ruby-vips#155 (comment) * Try it here
|
I was hitting this issue with puma workers (fork) and was able to resolve it with the following in my puma.rb: before_fork do
# ensure native libraries are initialized before forking
if RUBY_PLATFORM =~ /darwin/
require 'vips'
end
end |
Because of libvips/ruby-vips#396 and libvips/ruby-vips#155 we'll identify in a fork to allow for forking of the server process in production and/or parrallel testing
After updating to OS X 10.13.3, I received the following error trying to
require 'vips':Some googling took me to this post, which explains that the error comes from “incorrect code that happened to work most of the time in the past.” I have no way of knowing whether the issue is in
ruby-vipsorffiorlibvipsitself, and I don't really have the time or knowledge to track it down. Using one of that page's suggested workarounds – settingOBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESin the environment – allowed me to loadruby-vipswithout issue and get on with my work, though I realize I'm just asking OS X to hang later if it encounters a deadlock instead of crashing the process now.I'm reporting it here so there's both an issue to track and a concise explanation and workaround for anyone else who happens to google this error in a
vipscontext.The text was updated successfully, but these errors were encountered: