Multi threading issue.

[bwinter]

Hey,

I have been running my code in a single thread without issue. But as soon as I run two threads problems arise.

Parse::ParseProtocolError
: unauthorized
/home/deploy/.rvm/gems/ruby-2.0.0-p195/gems/parse-ruby-client-0.1.15/lib/parse/client.rb:106:in `request'
/home/deploy/.rvm/gems/ruby-2.0.0-p195/gems/parse-ruby-client-0.1.15/lib/parse/object.rb:120:in `save'
/home/deploy/gametime-backend/lib/stubhub_page.rb:24:in `putsSh'
/home/deploy/gametime-backend/lib/stubhub_page/login.rb:17:in `block in process'
/home/deploy/gametime-backend/lib/stubhub_page/login.rb:16:in `each'
/home/deploy/gametime-backend/lib/stubhub_page/login.rb:16:in `process'
/home/deploy/gametime-backend/lib/stubhub.rb:48:in `block (2 levels) in submit'
/home/deploy/gametime-backend/lib/stubhub.rb:42:in `each'
/home/deploy/gametime-backend/lib/stubhub.rb:42:in `block in submit'
/home/deploy/gametime-backend/lib/stubhub.rb:40:in `each'
/home/deploy/gametime-backend/lib/stubhub.rb:40:in `submit'
/home/deploy/gametime-backend/app/purchase_flow.rb:73:in `block (2 levels) in <top (required)>'

Occasionally I get the following error, which seems related:

WARN -- : Retrying Parse Error #<Patron::TimeoutError: easy handled already used in multi handle> on request /1/classes/PurchaseFlow nil "{...}]}}" response nil
...
[2013-07-17 18:40:29] ERROR Patron::Error: easy handled already used in multi handle
/home/bwinter/.rvm/gems/ruby-2.0.0-head/gems/patron-0.4.18/lib/patron/session.rb:223:in `handle_request'

I think it's unrelated but occasionally it segfaults as well (I think this is a selenium issue). The full error is massive so I will supply that upon request:

/home/bwinter/.rvm/gems/ruby-1.9.3-p429/gems/childprocess-0.3.9/lib/childprocess/abstract_process.rb:138: [BUG] Segmentation fault

I am not quite sure what's causing the issue here. I create my parse object then create a new thread that starts using it. If I remove every 'parse.save' in my code I can run multiple threads w/o issue. So I can only assume Parse is somehow causing these issues. Maybe the shared application_id/api_keys...

Parse.init(:application_id => "...", :api_key => "...")
parse = Parse::Object.new("PurchaseFlow", initial_data).save
Thread.new{ Processor.new.submit(parse) }

As you can see from my stack traces I have tried quite a few versions of ruby (1.9.3, 2.0.0-193p,247p,270p), none of them seemed to help.

Gemfile:

    braintree (2.23.0)
      builder (>= 2.0.0)
    builder (3.2.2)
    childprocess (0.3.9)
      ffi (~> 1.0, >= 1.0.11)
    excon (0.16.10)
    ffi (1.9.0)
    iron_core (0.6.2)
      rest (>= 2.2.0)
    iron_mq (4.0.3)
      iron_core (>= 0.5.1)
    json (1.7.7)
    jwt (0.1.8)
      multi_json (>= 1.5)
    kgio (2.8.0)
    mandrill-api (1.0.37)
      excon (~> 0.16.0)
      json (~> 1.7.7)
    mime-types (1.23)
    multi_json (1.7.7)
    net-http-persistent (2.8)
    parse-ruby-client (0.1.15)
      iron_mq
      patron
    patron (0.4.18)
    rack (1.5.2)
    rack-protection (1.5.0)
      rack
    raindrops (0.11.0)
    rest (2.6.3)
      net-http-persistent
      rest-client (>= 0.3.0)
    rest-client (1.6.7)
      mime-types (>= 1.16)
    rmagick (2.13.2)
    rubyzip (0.9.9)
    selenium-webdriver (2.33.0)
      childprocess (>= 0.2.5)
      multi_json (~> 1.0)
      rubyzip
      websocket (~> 1.0.4)
    sinatra (1.4.3)
      rack (~> 1.4)
      rack-protection (~> 1.4)
      tilt (~> 1.3, >= 1.3.4)
    tilt (1.4.1)
    twilio-ruby (3.9.0)
      builder (>= 2.1.2)
      jwt (>= 0.1.2)
      multi_json (>= 1.3.0)
    unicorn (4.6.3)
      kgio (~> 2.6)
      rack
      raindrops (~> 0.7)
    websocket (1.0.7)
    zbar (0.2.2)
      ffi (>= 1.0.0)

If there is any missing information that might help, let me know.

Thanks,
Brendan

[bwinter]

...?

parse-ruby-client-0.1.15/lib/parse/client.rb :

          #while true
          #  if @queue.reload.size >= @max_concurrent_requests
          #    sleep 1
          #  else
              # add to queue before request
              @queue.post("1")
              response = @session.request(method, uri, {}, options)
              # delete from queue after request
              msg = @queue.get()
              msg.delete
          #  end
          #end

[adelevie]

Is this causing problems?

Sent from my iPhone

On Jul 18, 2013, at 5:13 PM, Brendan Winter notifications@github.com wrote:

...?

parse-ruby-client-0.1.15/lib/parse/client.rb :

      #while true
      #  if @queue.reload.size >= @max_concurrent_requests
      #    sleep 1
      #  else
          # add to queue before request
          @queue.post("1")
          response = @session.request(method, uri, {}, options)
          # delete from queue after request
          msg = @queue.get()
          msg.delete
      #  end
      #end

—
Reply to this email directly or view it on GitHub.

[bwinter]

Not sure, but it looks like that was written to allow multiple concurrent connections. But seeing as it's not actually used and I am having concurrency issues ... it have to wonder if they aren't related.

[adelevie]

A while ago I tried to build a throttler for concurrent connections, but my use case invoked multiple clients, so I had no threading issues to worry about. Notice that it uses IronCache, which is remote storage.

I wish I could be more helpful re threading, but right now I lack the expertise to make this suit your needs.

Sent from my iPhone

On Jul 18, 2013, at 6:16 PM, Brendan Winter notifications@github.com wrote:

Not sure, but it looks like that was written to allow multiple concurrent connections. But seeing as it's not actually used and I am having concurrency issues ... it have to wonder if they aren't related.

—
Reply to this email directly or view it on GitHub.

[bwinter]

Not to worry, we tried mutexes to synchronize all the saves but that didn't seem to work - which is really strange.

I think we are going to change our approach and create a process dedicated to updating Parse. It's a little bit circuitous but should solve the problem.

Thanks for getting back to me.

Not sure if we should close this thread...?

[adelevie]

Can you close it? Feel free to repoen of you need.

Sent from my iPhone

On Jul 18, 2013, at 7:47 PM, Brendan Winter notifications@github.com wrote:

Not to worry, we tried mutexes to synchronize all the saves but that didn't seem to work - which is really strange.

I think we are going to change our approach and create a process dedicated to updating Parse. It's a little bit circuitous but should solve the problem.

Thanks for getting back to me.

Not sure if we should close this thread...?

—
Reply to this email directly or view it on GitHub.

[bwinter]

Closed for now - Trying new approach.

[ivanacostarubio]

I am in the same situation as @bwinter. I've been running some code single threaded for a few months without any problems. Recently, we move some of that code to use http://sidekiq.org/ and we are seeing some error for the first time. They are:

Parse::ParseProtocolError: : unauthorized
Patron::HostResolutionError
Patron::URLFormatError

Not sure if all the exceptions are related to thread safety. I'll investigate further.

😄

[rhymes]

Can you guys please take a look at #165 ?