operations like getpoint or stats are stalled when using them from irb

[stoicdavid]

Hi
To report that when I call getpoint or stats in the irb console, the console stalls and need to interrupt with ctrl-z

I load an image with Active Storage:

p = Picture.first
im = Vips::Image.new_from_file ActiveStorage::Blob.service.send(:path_for, p.image.key), access: :sequential

then I just call im.getpoint(0,0) and the console gets blocked.
Also if I call im.stats I obtain the same behavior

My dev stack is:

• Libvips 8.11
• Rails 6.1.3.2
• Ruby-vips 2.1.2
• working on zsh
• using Apple Silicon

[jcupitt]

Hi @stoicdavid, I'm sorry you're having trouble.

It sounds like a weird one -- it's fine for me. Unless there's some way I can reproduce this I'm not sure I can help.

john@banana ~ $ irb
irb(main):001:0> require 'vips'
=> true
irb(main):002:0> x = Vips::Image.new_from_file "/home/john/pics/k2.jpg", access: :sequential
=> #<Image 1450x2048 uchar, 3 bands, srgb>
irb(main):003:0> x.getpoint(0, 0)
=> [11.0, 1.0, 0.0]
irb(main):004:0> 

john@banana ~ $ gem info ruby-vips

*** LOCAL GEMS ***

ruby-vips (2.1.2, 2.1.1, 2.1.0)
    Author: John Cupitt
    Homepage: http://github.com/libvips/ruby-vips
    License: MIT
    Installed at (2.1.2): /home/john/packages/gems
                 (2.1.1): /home/john/packages/gems
                 (2.1.0): /home/john/packages/gems

    A fast image processing library with low memory needs
john@banana ~ $ irb --version
irb 1.2.6 (2020-09-14)
john@banana ~ $ ruby --version
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux-gnu]
john@banana ~ $ vips --version
vips-8.11.1-Wed Jun 23 11:25:51 UTC 2021

[jcupitt]

There is a known issue in 8.11.0 with modules. You could try disabling them -- change this line in vips.rb:

    system "./configure", *args

To be:

    system "./configure", "--disable-modules", *args

Then do:

brew uninstall vips
brew install --build-from-source  vips

[jcupitt]

Though I don't know if that'll fix the issue you're seeing.

[stoicdavid]

Thanks will try now and let you know

[stoicdavid]

Thanks @jcupitt, my issue appears in the rails console, I tested directly using an irb session using your same example and worked!
Then I thought it has something to do with the rails console and noticed that Spring preloader is the one causing the issue, I disabled the gem in my gemfile and then re-tried getpoint and stats in the rails console and worked.

I tried the installation disabling modules but I observed the same behavior in the rails console, so the module issue is unrelated.

I am checking now what is going on with Spring, I've recently upgrade into rails 6.1.3.2 will try with previous versions of rails and see if that works, also just noticed that rails released a newer version 6.1.4 so perhaps that might fix things. Will let you know...

[jcupitt]

Ah then I expect it's a fork problem.

You can't mix fork and threads in the same process, or at least only with great care. After a fork(), the parent stays in the same state (all running threads still there, all locks and signals still working), but the child starts with just one thread and all the thread structures (locks etc.) in whatever state they were at the time of the fork. So if a mutex is locked when the fork happens, the child will block and never unblock when it hits it.

Rails servers starts up everything, then when a request comes in, forks a child process to handle it. If ruby-vips is initialised at server startup, then the child process will not work. You need to switch off the pre-fork system, or only init ruby-vips when you start to handle a request.

This is true for any gem which uses threads of course, not just ruby-vips.

[jcupitt]

Yes, spring will do this kind of fork-per-request trickery, you'll need to check the configuration there.

[stoicdavid]

Thanks, will do some additional research later today and if I find something additional will post here for reference and close the issue. Many thanks

[jcupitt]

I think this is fixed in 8.11.3. Sorry for the regression.

[stoicdavid]

Thanks! tested it and it works!
For the sake of curiosity what was causing the issue? was it something related with the Spring preloader?

[jcupitt]

That's great! Thanks for testing, @stoicdavid.

The problem was that 8.11 has a new threading system, it was being started up on vips_init(), and that was being called on process start. Some time later, spring would fork the process to become a request handler, and all thread state breaks on fork.

The fix was just to delay threadpool creation until first use.