Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segfault occurs as soon as requires sorted_set or rbtree #1334

Closed
kimyu92 opened this issue Apr 25, 2022 · 11 comments
Closed

Segfault occurs as soon as requires sorted_set or rbtree #1334

kimyu92 opened this issue Apr 25, 2022 · 11 comments

Comments

@kimyu92
Copy link

kimyu92 commented Apr 25, 2022 •
edited

Description

As soon as, I installed both rmagick and rbtree or rmagick with sorted_set, segmentation fault would happen immediately with the single file rails setup below.

-- Control frame information -----------------------------------------------
c:0066 p:---- s:0334 e:000333 CFUNC  :read
c:0065 p:0050 s:0329 e:000328 METHOD /usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/processing/rmagick.rb:352
c:0064 p:0018 s:0317 e:000316 METHOD /usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/processing/rmagick.rb:176
c:0063 p:0050 s:0311 e:000310 BLOCK  /usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/processing.rb:83 [FINISH]
c:0062 p:---- s:0305 e:000304 CFUNC  :each
c:0061 p:0009 s:0301 e:000300 BLOCK  /usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/processing.rb:75
c:0060 p:0016 s:0298 e:000297 METHOD /usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/callbacks.rb:15
c:0059 p:0018 s:0292 e:000291 METHOD /usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/processing.rb:74
c:0058 p:0010 s:0287 e:000286 BLOCK  /usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/callbacks.rb:14 [FINISH]
c:0057 p:---- s:0283 e:000282 CFUNC  :each

...

-- Ruby level backtrace information ----------------------------------------
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:24:in `block in autorun'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:38:in `perform_at_exit'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:45:in `invoke'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:71:in `run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:89:in `run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:115:in `run_specs'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/reporter.rb:74:in `report'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:116:in `block in run_specs'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/configuration.rb:2068:in `with_suite_hooks'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:121:in `map'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example_group.rb:607:in `run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example_group.rb:642:in `run_examples'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example_group.rb:642:in `map'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example_group.rb:646:in `block in run_examples'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:259:in `run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:511:in `with_around_and_singleton_context_hooks'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:468:in `with_around_example_hooks'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:486:in `run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:629:in `run_around_example_hooks_for'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:352:in `call'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:390:in `execute_with'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:457:in `instance_exec'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:457:in `instance_exec'
/usr/local/bundle/gems/rspec-rails-5.1.2/lib/rspec/rails/adapters.rb:75:in `block (2 levels) in <module:MinitestLifecycleAdapter>'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:352:in `call'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:626:in `block in run_around_example_hooks_for'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:486:in `block in run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:468:in `block in with_around_example_hooks'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:511:in `block in with_around_and_singleton_context_hooks'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:261:in `block in run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:505:in `run_before_example'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:484:in `run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:614:in `run_example_hooks_for'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:614:in `reverse_each'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:615:in `block in run_example_hooks_for'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:528:in `run_owned_hooks_for'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:528:in `each'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:529:in `block in run_owned_hooks_for'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/hooks.rb:365:in `run'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:457:in `instance_exec'
/usr/local/bundle/gems/rspec-core-3.11.0/lib/rspec/core/example.rb:457:in `instance_exec'
repro.rb:69:in `block (2 levels) in <main>'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/orm/activerecord.rb:75:in `avatar='
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/mount.rb:373:in `avatar='
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/mount.rb:146:in `avatar='
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/mounter.rb:46:in `cache'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/mounter.rb:46:in `map'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/mounter.rb:47:in `block in cache'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/mounter.rb:176:in `handle_error'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/mounter.rb:63:in `block (2 levels) in cache'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/cache.rb:144:in `cache!'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/callbacks.rb:14:in `with_callbacks'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/callbacks.rb:14:in `each'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/callbacks.rb:14:in `block in with_callbacks'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/processing.rb:74:in `process!'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/callbacks.rb:15:in `with_callbacks'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/processing.rb:75:in `block in process!'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/processing.rb:75:in `each'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/uploader/processing.rb:83:in `block (2 levels) in process!'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/processing/rmagick.rb:176:in `resize_to_fit'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/processing/rmagick.rb:352:in `manipulate!'
/usr/local/bundle/gems/carrierwave-2.2.2/lib/carrierwave/processing/rmagick.rb:352:in `read'

Steps to Reproduce

  1. Pull either docker pull ruby:2.7.6-alpine3.15 or docker pull ruby:3.0.4-alpine3.15 or docker pull ruby:3.1.2-alpine3.15
  2. Save this file as repro.rb in container
  3. Run command to install imagemagick 7, apk add pkgconfig imagemagick imagemagick-dev imagemagick-libs
  4. Run command to install sqlite3 apk add sqlite sqlite-dev
  5. Run command to install other sys libs apk add build-base gcc autoconf libffi-dev openssl-dev
  6. Run ruby repro.rb inside docker container
# frozen_string_literal: true

require 'bundler/inline'

gemfile(true) do
  source 'https://rubygems.org'

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem 'rails', '~> 7.0.2.3'
  gem 'rspec-rails', '~> 5.1.1'

  gem 'carrierwave'
  gem 'rmagick', '~> 4.2.5'

  # either gem would make interpreter blows up
  # adding , require: false on either gem would make it work but not ideal
  # gem 'rbtree'
  gem 'sorted_set'
  gem 'sqlite3'
end

require 'rmagick'

require 'active_support'
require 'active_support/core_ext/object/blank'
require 'active_record'

require 'action_controller/railtie'
require 'action_view/railtie'

require 'rspec/autorun'
require 'rspec/rails'
require 'carrierwave/orm/activerecord'

ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: ':memory:')

ActiveRecord::Schema.define do
  drop_table :users, if_exists: true

  create_table :users, force: true do |t|
    t.timestamps null: false
    t.string :avatar
  end
end

class AvatarUploader < CarrierWave::Uploader::Base
  include CarrierWave::RMagick
  storage :file

  process resize_to_fit: [777, 777]
end

class User < ActiveRecord::Base
  mount_uploader :avatar, AvatarUploader
end

RSpec.configure do |config|
  config.filter_run_excluding skip: true
  config.run_all_when_everything_filtered = true
  config.filter_run focus: true
end

RSpec.describe 'Segfault error' do
  let(:user) { User.new }
  let(:file) { Tempfile.new }
  let(:avatar) { Rack::Test::UploadedFile.new(file, 'application/pdf', true, original_filename: 'sample.pdf') }
  let(:avatar_key) { sample_key(User.new.uploader, extension: 'pdf') }

  before do
    user.avatar = avatar
  end

  it { expect(user.avatar).to eq(avatar) }
end

System Configuration

  • ImageMagick version: 7.1.0-16 Q16-HDRI x86_64
  • RMagick version: 4.2.5
  • Ruby version: 2.7.6/3.0.4/3.1.2
  • Environment (Operating system, version and so on): alpine 3.15.x (official ruby image in docker hub, https://hub.docker.com/_/ruby?tab=tags)
  • Additional information:
@kimyu92 kimyu92 changed the title Segfault occurs as soon as include sorted_set or rbtree Segfault occurs as soon as requires sorted_set or rbtree Apr 25, 2022
@kimyu92
Copy link
Author

kimyu92 commented Apr 26, 2022

@Watson1978 please let me know if you have any insight especially this is most likely involve C extension 🤷

@kyrylo kyrylo mentioned this issue Apr 26, 2022
Open
@kimyu92
Copy link
Author

kimyu92 commented Apr 26, 2022

Seems like it's suggesting this line
https://github.com/carrierwaveuploader/carrierwave/blob/v2.2.2/lib/carrierwave/processing/rmagick.rb#L352

and taking closer look Magick::Image.read is defined in https://github.com/rmagick/rmagick/blob/main/ext/RMagick/rmmain.c#L294

@dlemstra Since you're very familiar with C code, any potential issue that you can share regarding how rbtree could interfere read?

@Watson1978
Copy link
Member

https://gist.github.com/Watson1978/0ec37694a363c64414b944ed47dd8373

$ docker build -t rmagick .

$ docker run -v $(pwd):/opt/rmagick --rm -it rmagick bash
bash-5.1# gdb --args ruby ./crash.rb 
GNU gdb (GDB) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ruby...
(gdb) r
Starting program: /usr/local/bin/ruby ./crash.rb

...

-- drop_table(:users, {:if_exists=>true})
   -> 0.0081s
-- create_table(:users, {:force=>true})
   -> 0.0009s
Run options:
  include {:focus=>true}
  exclude {:skip=>true}

All examples were filtered out; ignoring {:focus=>true}

Thread 1 "ruby" received signal SIGSEGV, Segmentation fault.
0x00007f2e6a2dc9d9 in dict_first () from /usr/local/lib/ruby/gems/3.0.0/gems/rbtree-0.4.5/lib/rbtree.so
(gdb) bt 20
#0  0x00007f2e6a2dc9d9 in dict_first () from /usr/local/lib/ruby/gems/3.0.0/gems/rbtree-0.4.5/lib/rbtree.so
#1  0x00007f2e67ade974 in ?? () from /usr/lib/libgs.so.9
#2  0x00007f2e67adebb6 in dict_resize () from /usr/lib/libgs.so.9
#3  0x00007f2e67aec4ae in ?? () from /usr/lib/libgs.so.9
#4  0x00007f2e67ae20d5 in ?? () from /usr/lib/libgs.so.9
#5  0x00007f2e67ae356b in gs_interpret () from /usr/lib/libgs.so.9
#6  0x00007f2e67ad7f2b in ?? () from /usr/lib/libgs.so.9
#7  0x00007f2e67ad834a in gs_main_init2aux () from /usr/lib/libgs.so.9
#8  0x00007f2e67ad8ce1 in gs_main_init2 () from /usr/lib/libgs.so.9
#9  0x00007f2e67ada10b in ?? () from /usr/lib/libgs.so.9
#10 0x00007f2e67ada3c5 in ?? () from /usr/lib/libgs.so.9
#11 0x00007f2e67adaaed in ?? () from /usr/lib/libgs.so.9
#12 0x00007f2e67adba50 in gs_main_init_with_args01 () from /usr/lib/libgs.so.9
#13 0x00007f2e67adbbb5 in gs_main_init_with_args () from /usr/lib/libgs.so.9
#14 0x00007f2e6b1072e0 in ?? () from /usr/lib/ImageMagick-7.1.0/modules-Q16HDRI/coders/pdf.so
#15 0x00007f2e6a7c45cc in ReadImage () from /usr/lib/libMagickCore-7.Q16HDRI.so.10
#16 0x00007f2e6a9c8687 in rd_image () from /usr/local/lib/ruby/gems/3.0.0/gems/rmagick-4.2.5/lib/RMagick2.so
#17 0x00007f2e6a9c84c8 in Image_read () from /usr/local/lib/ruby/gems/3.0.0/gems/rmagick-4.2.5/lib/RMagick2.so
#18 0x0000564a86cf5482 in call_cfunc_1 ()
#19 0x0000564a86cf6a45 in vm_call_cfunc_with_frame ()
(More stack frames follow...)

hmm, I confirmed that your code causes SEGV within rbtree.
I will investigate this deeply at weekend

@Watson1978
Copy link
Member

Watson1978 commented Apr 26, 2022 •
edited

If possible, it would be better to use something other than PDF, since processing JPEG instead of PDF does not seem to cause SEGV.

/usr/lib/libgs.so.9 is related to GhostScript.
I don't know why it calls rbtree...

@kimyu92
Copy link
Author

kimyu92 commented Apr 26, 2022

hmm, I confirmed that your code causes SEGV within rbtree.
I will investigate this deeply at weekend

Thank you

I don't know why it calls rbtree

💯 Shouldn't rmagick and rbtree be isolated unless c extension clashed with each other via include? 😅

@Watson1978
Copy link
Member

Watson1978 commented Apr 27, 2022 •
edited

A simpler reproduction code would look like below.

require 'bundler/inline'

gemfile(true) do
  source 'https://rubygems.org'

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem 'rmagick', '~> 4.2.5'
  gem 'sorted_set'
end

require 'rmagick'
require 'sorted_set'

Magick::Image.read('sample.pdf')

@Watson1978
Copy link
Member

I'm guessing there is something wrong with the alpine configuration or something, and the loaded C symbols are conflicting....

@kimyu92
Copy link
Author

kimyu92 commented Apr 27, 2022

I'm guessing there is something wrong with the alpine configuration or something, and the loaded C symbols are conflicting....

I'm curious would it be due to dict_first() from ghostscript vs dict_first() defined by rbtree

@Watson1978
Copy link
Member

Watson1978 commented Apr 27, 2022 •
edited

At least, there is no way that can be solved with RMagick.

(It stop using PDF or create a docker image with another OS instead of alpine)

@kimyu92
Copy link
Author

kimyu92 commented Apr 27, 2022

@Watson1978 I highly appreciated the gist above and I agreed with you that this is not related to rmagick. musl is usually stricter than glibc, I still lean towards the issue might come from rbtree because without including it, the rmagick works fine.

@kimyu92 kimyu92 closed this as completed Apr 27, 2022
@Watson1978
Copy link
Member

I think the rbtree and ghostscript libraries are not linked, so it is curious that the symbols conflict.
And because your reproduction code works on ubuntu image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Watson1978 @kimyu92