Every repository with this icon (
) is private.
Every repository with this icon (
) is public.
| Description: | self assembling fabric of ruby daemons edit |
-
1 vote
Mapper unable to send requests from Merb (mongrel adapter)
2 comments Created 3 months ago by highandwildI have started the mapper in config/init.rb but Nanite.request() simply returns false without any error message. The agent does not receive the request, either.
But it works fine if I test it with nanite-mapper !
Here's the code: http://gist.github.com/147641
ruby - 1.8.7
mongrel - 1.1.5
nanite - 0.4.1 (edge)
amqp - 0.6.0
rabbitmq - 1.5.5
Arch - x64
OS - Cent OS 5.1 (virtual box)Comments
-
Getting a weird SSL error with ruby 1.9r129 on osx - any help / ideas would be appreciated!
$ rake spec /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:31: [BUG] Bus Error ruby 1.9.1p129 (2009-05-12 revision 23412) [i386-darwin9.7.0] -- control frame ---------- c:0029 p:---- s:0080 b:0080 l:000079 d:000079 CFUNC :initialize c:0028 p:---- s:0078 b:0078 l:000077 d:000077 CFUNC :new c:0027 p:0063 s:0075 b:0075 l:000074 d:000074 CLASS /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:31 c:0026 p:0011 s:0073 b:0073 l:000072 d:000072 CLASS /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:23 c:0025 p:0011 s:0071 b:0071 l:000070 d:000070 CLASS /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:22 c:0024 p:0045 s:0069 b:0069 l:000068 d:000068 TOP /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:21 c:0023 p:---- s:0067 b:0067 l:000066 d:000066 FINISH c:0022 p:---- s:0065 b:0065 l:000064 d:000064 CFUNC :require c:0021 p:0059 s:0061 b:0061 l:000060 d:000060 TOP /usr/local/lib/ruby19/site_ruby/1.9.1/openssl.rb:22 c:0020 p:---- s:0059 b:0059 l:000058 d:000058 FINISH c:0019 p:---- s:0057 b:0057 l:000056 d:000056 CFUNC :require c:0018 p:0083 s:0053 b:0053 l:000052 d:000052 TOP /Users/ippy04/Code/Samples/nanite/lib/nanite.rb:7 c:0017 p:---- s:0051 b:0051 l:000050 d:000050 FINISH c:0016 p:---- s:0049 b:0049 l:000048 d:000048 CFUNC :require c:0015 p:0084 s:0045 b:0045 l:000044 d:000044 TOP /Users/ippy04/Code/Samples/nanite/spec/spec_helper.rb:6 c:0014 p:---- s:0043 b:0043 l:000042 d:000042 FINISH c:0013 p:---- s:0041 b:0041 l:000040 d:000040 CFUNC :require c:0012 p:0039 s:0037 b:0037 l:000036 d:000036 TOP /Users/ippy04/Code/Samples/nanite/spec/actor_registry_spec.rb:1 c:0011 p:---- s:0035 b:0035 l:000034 d:000034 FINISH c:0010 p:---- s:0033 b:0033 l:000032 d:000032 CFUNC :load c:0009 p:0012 s:0029 b:0029 l:000020 d:000028 BLOCK /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/example_group_runner.rb:15 c:0008 p:---- s:0026 b:0026 l:000025 d:000025 FINISH c:0007 p:---- s:0024 b:0024 l:000023 d:000023 CFUNC :each c:0006 p:0036 s:0021 b:0021 l:000020 d:000020 METHOD /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/example_group_runner.rb:14 c:0005 p:0097 s:0017 b:0017 l:000016 d:000016 METHOD /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/options.rb:107 c:0004 p:0068 s:0012 b:0012 l:000011 d:000011 METHOD /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/command_line.rb:9 c:0003 p:0077 s:0007 b:0006 l:002614 d:002494 EVAL /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/bin/spec:4 c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH c:0001 p:0000 s:0002 b:0002 l:002614 d:002614 TOP :43044 --------------------------- -- Ruby level backtrace information----------------------------------------- /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:31:in `initialize' /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:31:in `new' /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:31:in `' /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:23:in `' /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:22:in `' /usr/local/lib/ruby19/site_ruby/1.9.1/openssl/ssl.rb:21:in `' /usr/local/lib/ruby19/site_ruby/1.9.1/openssl.rb:22:in `require' /usr/local/lib/ruby19/site_ruby/1.9.1/openssl.rb:22:in `' /Users/ippy04/Code/Samples/nanite/lib/nanite.rb:7:in `require' /Users/ippy04/Code/Samples/nanite/lib/nanite.rb:7:in `' /Users/ippy04/Code/Samples/nanite/spec/spec_helper.rb:6:in `require' /Users/ippy04/Code/Samples/nanite/spec/spec_helper.rb:6:in `' /Users/ippy04/Code/Samples/nanite/spec/actor_registry_spec.rb:1:in `require' /Users/ippy04/Code/Samples/nanite/spec/actor_registry_spec.rb:1:in `' /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/example_group_runner.rb:15:in `load' /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/example_group_runner.rb:15:in `block in load_files' /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/example_group_runner.rb:14:in `each' /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/example_group_runner.rb:14:in `load_files' /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/options.rb:107:in `run_examples' /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib/spec/runner/command_line.rb:9:in `run' /usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/bin/spec:4:in `' -- C level backtrace information ------------------------------------------- 0x117042 0 ruby19 0x00117042 rb_vm_bugreport + 82 0x2c21c 1 ruby19 0x0002c21c rb_warning + 444 0x2c27b 2 ruby19 0x0002c27b rb_bug + 43 0xbd37b 3 ruby19 0x000bd37b rb_enable_interrupt + 75 0x91e0b2bb 4 libSystem.B.dylib 0x91e0b2bb _sigtramp + 43 0xffffffff 5 ??? 0xffffffff 0x0 + 4294967295 [NOTE] You may encounter a bug of Ruby interpreter. Bug reports are welcome. For details: http://www.ruby-lang.org/bugreport.html rake aborted! Command /usr/local/bin/ruby19 -I"/usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/lib" "/usr/local/lib/ruby19/gems/1.9.1/gems/rspec-1.2.7/bin/spec" "spec/actor_registry_spec.rb" "spec/actor_spec.rb" "spec/agent_spec.rb" "spec/cached_certificate_store_proxy_spec.rb" "spec/certificate_cache_spec.rb" "spec/certificate_spec.rb" "spec/cluster_spec.rb" "spec/dispatcher_spec.rb" "spec/distinguished_name_spec.rb" "spec/encrypted_document_spec.rb" "spec/job_spec.rb" "spec/local_state_spec.rb" "spec/log_spec.rb" "spec/mapper_spec.rb" "spec/packet_spec.rb" "spec/rsa_key_pair_spec.rb" "spec/secure_serializer_spec.rb" "spec/serializer_spec.rb" "spec/signature_spec.rb" "spec/static_certificate_store_spec.rb" "spec/util_spec.rb" --format specdoc --colour failed
Comments
pmamediagroup
Tue Jul 28 16:02:23 -0700 2009
| link
same problem with 1.8.7
-
0 votes
Bug in mapper.rb where options are passed as wrong parameter (size)
1 comment Created 2 months ago by joshwilsdonIn mapper.rb the request function looks like:
request(type, payload = '', opts = {}, &blk)and then passes the first 3 arguments in the same order to:
build_deliverable(deliverable_type, type, payload, opts)with deliverable_type == Request. Then build_deliverable passes these arguments directly to:
deliverable_type.new(type, payload, opts)but the problem is that Request has an initialize function that looks like:
initialize(type, payload, size=nil, opts={})so the options that were passed in to the original request get passed in as the 'size' parameter, which obviously doesn't work. This causes any :selector or :target to be ignored for example.
In our environment, this was causing problems when we passed the :target option because it was being ignored. Changing the build_deliverable function to make the call:
deliverable_type.new(type, payload, nil, opts)fixed the issue.
Comments
-
0 votes
load average / status function not updated with heartbeat messages when using Redis
1 comment Created 2 months ago by joshwilsdonWhen using redis the nanite- key gets set with the load_average when the node registers but it is not updated after that by heartbeat messages. It seems that the intention is that the heartbeat/ping sends the load average so that the load average will be put in Redis. This is not happening because the code in cluster.rb (handle_ping) is doing:
if nanite = nanites[ping.identity] nanite[:status] = ping.statusbut nanites[ping.identity] returns an anonymous Hash, so updating it here does nothing. As such this value is never sent to Redis. I have confirmed that hacking in a update_status function to the Nanite::State class (which just updates the nanite- key in redis) and then calling it in the handle_ping as:
nanites.update_status(ping.identity, ping.status)causes the value in Redis to be updated at every heartbeat. Was it the intended behavior for this function (handle_ping) to update Redis? The comment seems to indicate that is the case.
Comments
-
Supposing cli-mapper that sends a push to a client-agent:
push('/client_agent/foo', "hi")And this client-agent sends a push (or request) to a main-agent:
push('/main_agent/foo', ["bar"] )Main agent receives 2 requests instead of one. If I have for instance, 8 thins running, main agent receives 10 exaclty equal requests.
Looks like the requests are getting multiplicated by each mapper online on that time.Comments
I'm guessing you're not using Redis, correct? If you don't all mappers will get the agent's request, and therefore all of them will forward it to an agent of their own. When using Redis only one mapper will get the request and forward it. It's probably a matter of discussion if this is a bug or a feature. I guess it wouldn't hurt to enable exclusiveness on the request queue when not using Redis as well, but there may be situations where not all mappers know the appropriate agents yet to handle that request. That again wouldn't hurt too much when using an offline queue, since that'd eventually lead to the right agent getting the message.
Hi Matt, thanks for the quick reply.
Yup, I'm not using redis. Hm, lots of options:
Use Redis, because this will really be a bug here this, heh...
Try to play with a tokyo adapter (already got it running on the server), redis is key/value store, right?
Or, if it's not too much to ask, could you give me some directions on how to disable this "feature" ? and rely on offline queue.Redis is key/value, that's correct. I don't have a solution for it per se. A quickfix would be to look into cluster.rb. In setup_request_queue it's using an exclusive queue when using Redis. You could try always using that, i.e. remove the shared_state? check. The offline queue is a simple command line switch (--offline-failsafe) for the nanite-agent and a parameter for the mapper class (:offline_failsafe => true). It's just an additional sanity check which you should do anyway when you're relying on your messages being delivered.
The worst case scenario should be avoidable when using the offline queue and only having one mapper pick up the message from the request queue. Let me know if that works. Maybe it's worth considering to get that fix in.
Ops, now I realize I was pvt messaging matt. Sorry man.
Same issue with redis enabled. =/
UPDATE: I've tryed editing setup_request_queue in all ways I could imagine, same result.
Btw, got 2 specs failiing too, something about the ProxyMapper instance was not being erased...That's a bit odd, even though we had to fix some issues with Redis and internal timeouts, that solved it for us, since then only one mapper gets the request and forwards it. I'd need some more log output from the mapper logs to get a better look at what's going on.
Gosh, I'm embarassed now, There was a mapper running I didn't see (w/o redis).
It's working fine with Redis. Really sorry, need to sleep, nanite is givin me some insomnia... (and it feels great).
Thank you Matt, I owe you a (or some) beer. Just let me know when you came to Brazil.Just to confirm, removing the -#{identity} and the exclusive option of the amq.queue request, it works. Only one request and without Redis.
I'll be happy to work in a patch to make this an "option", if anyone is interested, or no better solution came to light. In the while, will keep a fork to make it easy to install it on my servers.Thanks matt, thanks all you nanite devs, you guys rock!
I've added an option to the mapper init, well, it works, will try in production this weekend.
http://github.com/nofxx/nanite/commit/7804058cf297088f063cf5d1d2695c8b15ab71a0Gonna write/fix the specs soon.. heh, sorry about the emacs whitespace cleanup too.
Wow, finally, looks like it's working now! ;)
It's only calling it once, offline_failsafe ensure that some agent will find the request. All good.Was having a weird problem with some actors that use ActiveRecord, they just stop advertising their methods. The problem was I didn't knew about single-threaded... working fine now.
I've added those stones I've found on my path to the wiki. Again, thanks!
Good to be on nanite hehe..Just an update: Albeit working flawless for weeks, on the deploy something strange happens (sometimes), the rails mappers, if I'm not wrong:
heartbeat-19018d26cdb64d27e25c55d007e73ebb 8149
heartbeat-25941f2a7e262e05e05c2349f08ff468 8153
....Heartbeats start to accumulate, until god restart RabbitMQ, than everything gets back to normal... heh weird.
But heartbeat is about to be gonne, right? heh -
0 votes
eventmachine not initialized: evma_send_data_to_connection (RuntimeError)
5 comments Created 11 days ago by leomayleomaySystem configurations:
ruby 1.8.7 (2008-08-11 patchlevel 72) [universal-darwin10.0]
amqp (0.6.0)
eventmachine (0.12.10)
rabbitmq 1.7.0
erlang 5.6.5I follow the steps given by the "Test Nanite (finally)",
First, I created a new agent called bob, http://gist.github.com/223056
Second, I created a new mapper in another shell, http://gist.github.com/223058
Comments
Could you please install EventMachine 0.12.8 and check if that works? I'd be curious if the problem is the just recently released 0.12.10 which we haven't tried with Nanite and the AMQP library yet.
leomayleomay
Sat Oct 31 06:22:47 -0700 2009
| link
thank you, it works
Pretty odd that it'd break with a patch release of EventMachine. Thanks for testing, I'll look into it.
The solution is to use the latest AMQP (version 0.6.5) gem, it's up on gemcutter and removes a custom patch for EventMachine. I'll update the documentation accordingly.
leomayleomay
Mon Nov 02 16:45:38 -0800 2009
| link
great info, thank you, mattmatt
-
I met some promblem when using nanite. I think it may caused by gem environment.
Can you tell me the gem version information which you guys running.One problem I met is.
After I start up agent and mapper using the simple-agent example which is in the source code, the agent side log stopped at :
[Fri, 06 Nov 2009 17:07:01 +0800] INFO: SEND [register] d72993a0de1aed09f42dd291e5046d3d, services: /simple/echo, /simple/time, /simple/gems, /simple/yielding, /simple/delayed, tags:And the mapper side the log stopped at:
[Fri, 06 Nov 2009 17:07:11 +0800] INFO: [setup] starting mapperafter a while the mapper down, cause this error
/opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:811:in `connect_server': no connection (RuntimeError)from /opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:811:in `reconnect' from /opt/local/lib/ruby/gems/1.8/gems/amqp-0.6.0/lib/amqp/client.rb:172:in `reconnect' from /opt/local/lib/ruby/gems/1.8/gems/amqp-0.6.0/lib/amqp/client.rb:85:in `call' from /opt/local/lib/ruby/gems/1.8/gems/amqp-0.6.0/lib/amqp/client.rb:85:in `unbind' from /opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:995:in `call' from /opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:995:in `run_deferred_callbacks' from /opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:995:in `times' from /opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:995:in `run_deferred_callbacks' from /opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:242:in `run_machine' from /opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:242:in `run' from simpleagent/cli.rb:21My environment is EventMachine 0.12.8 and amqp 0.6.0, rabbitmq 1.7.1
I just think whether it is rabbitmq permission,I try to re-install rabbitmq and run the "rabbitconf.rb" , then I the problem still.
here is the log when I run "rabbitconf.rb"Setting permissions for user "mapper" in vhost "/nanite" ...
...done. Setting permissions for user "nanite" in vhost "/nanite" ... ...done. Listing users ... guest
mapper
nanite
...done. Listing vhosts ... / /nanite ...done. Listing permissions in vhost "/nanite" ... mapper . . .
nanite . . .
...done.Comments
Can you be a bit more specific? Do you mean the installed dependencies of Nanite? Do you get a specific error message at some point while trying it out?
If so, you need either EventMachine 0.12.8 and the amqp gem < 0.6.5 or EventMachine 0.12.10 and the amqp gem = 0.6.5.
-
Nanite depends on the json gem, this should be defined in the gemspec.
Comments
Update: I have no idea how (perhaps because I downgraded to ruby 1.8.6) but it seems to finally queue the job. Now the problem is that it just queues it and it never reaches the agent ! (I'm running it inside a merb console).
Nanite.mapper.job_warden.inspect shows this:
And yes, I am getting a hearbeat from the agents. Also, Nanite.mapper.cluster.nanites lists the nanites !
What am I doing wrong ?
Tested it with sinatra (mongrel), and it works perfectly. Also upgraded to amqp edge. Still Zero luck with merb :( I also ran merb in debug mode but can't find anything amiss...