Problem with sidekiq and chromedriver combined

[brunotiago]

Hello,

First of all I don't know if this is the right place to ask for help on this, but I will try.

I have a rails app which is trying to launch a chromedriver to navigate and parse a website. The thing is: when I call the code from rails console it works pretty fine, but when I add a job to process on sidekiq and it is processed from there, the exact same code doesn't work and I have an error: Selenium::WebDriver::Error::WebDriverError: unable to connect to /usr/bin/chromedriver 127.0.0.1:9515 and the job fails to execute. I printed env vars, current user, chromedriver path, google-chrome, etc; and everything seems exactly the same. Right now I'm out of the ideas of how to proceed and debug. I don't know a lot about sidekiq but I believe this is related with some context/environment on Sidekiq or the way Sidekiq is spawning processes. It's just a guessing from me, but I don't know how to debug that or change the code.

I will attach part of the code for you to see and try to give me some lights on this.

module Requester
  PROXY_ENABLED = true
  HEADLESS = true
  USER_AGENT = 'Mozilla/5.0 (X11;'

  def proxy_enabled?
    PROXY_ENABLED
  end

  def proxy_username
    ProxyManager.instance.username
  end

  def proxy_password
    ProxyManager.instance.password
  end

  def random_proxy
    ProxyManager.instance.random_proxy(self.class::SOURCE_NAME)
  end

  def user_agent
    USER_AGENT
  end

  def build_selenium
    proxy_addr = proxy_enabled? ? random_proxy : nil

    proxy = proxy_enabled? ? Selenium::WebDriver::Proxy.new(
      http: proxy_addr,
      ssl: proxy_addr
    ) : nil

    options = Selenium::WebDriver::Chrome::Options.new(
      proxy: proxy
    )

    options.add_argument('--no-sandbox')
    options.add_argument('--user-agent=#{user_agent}')
    options.add_argument('--disable-crash-reporter')
    options.add_argument('--headless=new') if HEADLESS
    options.add_argument('--disable-ipv6')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument('--ignore-certificate-errors')
    options.add_argument('--window-size=1920,1080')
    options.add_argument('--enable-logging')
    options.add_argument('--log-path=~/chromedriver.log')


    driver = Selenium::WebDriver.for(:chrome, options: options)

    if proxy_enabled?
      driver.register(username: proxy_username, password: proxy_password)
    end

    # Return a hash containing both the driver, the proxy address, and the source name
    {
      driver: driver,
      proxy_addr: proxy_addr,
      source_name: self.class::SOURCE_NAME
    }
  end
end