Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Timeout::Error execution expired when host down in replica set (again) #126

Closed
telent opened this Issue Dec 10, 2012 · 0 comments

Comments

Projects
None yet
2 participants
Contributor

telent commented Dec 10, 2012

The fix in #112 doesn't fix the issue, for me. It seems that #connect can be called other than from #refresh

The underlying issue seems to be that in code of the form

Timeout::timeout { begin ; do_stuff; rescue Timeout::Error ; end }

the timeout is raised from outside the begin/rescue block and so will never be caught by it. You can verify this in irb by doing e.g.

1.9.1 :090 > s=Moped::Session.new(["10.10.6.23:27017",'10.10.5.196:27017','10.10.7.74:27017'], consistency: :strict, safe: true, database: 'admin').tap {|s| s.login('mongoadmin','my admin password') }
1.9.1 :103 > sock=s.cluster.nodes.first.send(:connection).instance_variable_get(:@sock)
1.9.1 :110 > Timeout::timeout(5) { sock.send(:handle_socket_errors) {sleep(10) }  }
Timeout::Error: execution expired

I would expect this to raise a Moped::Errors::ConnectionFailure which in real code would then then be caught by ensure_connected which would retry the connection. Instead it raises Timeout::Error

Here's the actual backtrace we hit in production when restarting the primary node. Note that this error is independent of whether SSL is enabled

A Timeout::Error occurred in background at 2012-12-03 23:30:59 +0000 :

  execution expired
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/tcp.rb:19:in `initialize'

  -------------------------------
Backtrace:
-------------------------------

  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/tcp.rb:19:in `initialize'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/tcp.rb:19:in `block in initialize'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/connectable.rb:78:in `handle_socket_errors'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/tcp.rb:19:in `initialize'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/connectable.rb:105:in `new'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/connectable.rb:105:in `block in connect'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/sockets/connectable.rb:104:in `connect'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/connection.rb:39:in `connect'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/connection.rb:201:in `with_connection'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/connection.rb:155:in `write'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:555:in `block (2 levels) in flush'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:129:in `ensure_connected'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:554:in `block in flush'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:569:in `logging'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:553:in `flush'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:542:in `process'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:363:in `query'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/session/context.rb:44:in `block in query'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/session/context.rb:109:in `block in with_node'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/cluster.rb:205:in `block in with_primary'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/node.rb:188:in `ensure_primary'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/cluster.rb:204:in `with_primary'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/session/context.rb:108:in `with_node'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/session/context.rb:43:in `query'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/moped-1.3.0/lib/moped/query.rb:114:in `first'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/mongoid-3.0.11/lib/mongoid/contextual/mongo.rb:201:in `first'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/gems/mongoid-3.0.11/lib/mongoid/contextual.rb:18:in `first'
  /home/sbopr/chopin/shared/bundle/ruby/1.9.1/bundler/gems/pooka-d29dfebea3b1/lib/pooka/analytica/pooka_worker.rb:31:in `find_object'

@durran durran closed this in e4fb18e Dec 24, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment