Jruby error

[oshanz]

below script works fine on MRI but fails sometime on Jruby

require 'ferrum'
require 'concurrent'


browser = Ferrum::Browser.new(process_timeout: 10, headless: false)
pool = Concurrent::FixedThreadPool.new(5)

context = browser.contexts.create

page = context.create_page
page.goto("https://www.google.com/search?q=Ruby+static+typing")
page.close

def extract_data(context)
    page = context.create_page
    page.goto("https://www.google.com/search?q=Ruby+static+typing")
    p page.at_xpath('/html/body/div[6]/div[3]/div[8]/div[1]/div[2]/div/div[2]/div[2]/div/div/div[1]/div/div[1]/a/h3').text
    page.close

end

100.times do 
    pool.post do
        begin
            extract_data(context)
        rescue Exception => e
            p e.message
        end
    end
end

pool.shutdown
pool.wait_for_termination

context.dispose
browser.quit

output

warning: thread "Ruby-0-Thread-3: /home/oshan/.asdf/installs/ruby/jruby-9.2.11.1/lib/ruby/gems/shared/gems/ferrum-0.8/lib/ferrum/browser/client.rb:19" terminated with exception (report_on_exception is true):
Concurrent::MultipleAssignmentError: Concurrent::MultipleAssignmentError
         set at /home/oshan/.asdf/installs/ruby/jruby-9.2.11.1/lib/ruby/gems/shared/gems/concurrent-ruby-1.1.6/lib/concurrent-ruby/concurrent/ivar.rb:114
  initialize at /home/oshan/.asdf/installs/ruby/jruby-9.2.11.1/lib/ruby/gems/shared/gems/ferrum-0.8/lib/ferrum/browser/client.rb:31

please run multiple attempts to see the errors.
I can see browser opening blank tabs. that cause to print Timed out waiting for response. It's possible that this happened because something took a very long time (for example a page load was slow). If so, setting the :timeout option to a higher value might help. and when I wasn't handling the exception rescue I got same warning as above. so I guess it relate to page close method?

[route]

JRuby support is not guaranteed because concurrent-ruby gem has now troubles on JRuby, if you are looking for a threaded solution you might be interested in https://github.com/rubycdp/vessel

[rajuvvadi]

Hey @route @oshanz , I have a Sidekiq Job that opens a website and navigates through 5 pages while filling input fields and checking radio/checkboxes.

I haven't gone through entire Ferrum code, so my patch might be incorrect or doesn't make sense. But, Here's what I did to possibly fix Concurrent::MultipleAssignmentError issue in my app.

  # file: lib/ferrum/browser/client.rb
  def initialize
    # .....
    # 
    while message = @ws.messages.pop
      if INTERRUPTIONS.include?(message["method"])
        @interruptor.async.call(message)
      elsif message.key?("method")
        @subscriber.async.call(message)
      else
        # here's the change
        retries_left = 3
        begin
          @pendings[message["id"]]&.set(message)
        rescue Concurrent::MultipleAssignmentError => e
          retries_left -= 1
          if retries_left > 0
            # retry after sleeping 200ms
            sleep(0.2)
            retry
          end
          raise e
        end
      end
    end
    # .....
  end

After this patch, I am yet to see any issues in my script. Plus, app doesn't throw that exception and terminate anymore.

[route]

@rajuvvadi I'm afraid it's only a hack, which doesn't solve the real issue