Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

can't enable plugin #3

Open
sonoman opened this Issue · 50 comments

4 participants

@sonoman

This is the output of the command line. rabbithub plug-in is expecting a plugin named "rabbitmq_mochiweb", which I can't find on my rabbitmq distribution. I've tried enabling similar plugins (see below), with no success.

root@ubuntu:/mnt/hgfs/Shared# sudo rabbitmq-plugins enable rabbithub
Error: The following dependencies could not be found:
rabbitmq_mochiweb
root@ubuntu:/mnt/hgfs/Shared# sudo rabbitmq-plugins enable mochiweb
Plugin configuration unchanged.
root@ubuntu:/mnt/hgfs/Shared# sudo rabbitmq-plugins enable webmachine
Plugin configuration unchanged.
root@ubuntu:/mnt/hgfs/Shared# sudo rabbitmq-plugins enable rabbithub
Error: The following dependencies could not be found:
rabbitmq_mochiweb

Is there a way to solve this ?

@tonyg
Owner

I'm afraid I haven't been keeping the rabbithub codebase up-to-date with respect to RabbitMQ. It probably needs some work to run with recent releases; in particular, to run with the current RabbitMQ HTTP-server APIs.

@sonoman sonoman closed this
@sonoman sonoman reopened this
@sonoman

It seems like replacing rabbitmq_mochiweb by "mochiweb", allowed me to enable the plugin. now I'm gonna try if it actually works.
BTW, are you planning to evolve this plugin ? or is this project kind of "idle" ?do you know if there is another alternative for working with webhooks ?

@tonyg
Owner
@brc859844
Collaborator
@sonoman

Sadly i got some errors during startup of the broker after enabling rabbithub. I tried recompiling (a recommendation i found in some blog for the same error on another component), but it got worst...that time it didn't even start the broker. Bad luck :(

=ERROR REPORT==== 8-Apr-2013::14:26:23 ===
Loading of /var/lib/rabbitmq/mnesia/rabbit@ubuntu-plugins-expand/rabbithub/ebin/
rabbithub_subscription.beam failed: badfile

=ERROR REPORT==== 8-Apr-2013::14:26:23 ===
beam/beam_load.c(1365): Error loading module rabbithub_subscription_sup:
use of opcode 153; this emulator supports only up to 152

=ERROR REPORT==== 8-Apr-2013::14:26:23 ===
Loading of /var/lib/rabbitmq/mnesia/rabbit@ubuntu-plugins-expand/rabbithub/ebin/
rabbithub_subscription_sup.beam failed: badfile

=ERROR REPORT==== 8-Apr-2013::14:26:23 ===
beam/beam_load.c(1365): Error loading module rabbithub_sup:
use of opcode 153; this emulator supports only up to 152

@brc859844
Collaborator
@sonoman

This is the output when I enter erlang's interactive shell:

root@ubuntu:/mnt/hgfs/Shared/aws-dev# erl
Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false]

Eshell V5.8.5 (abort with ^G)

@sonoman

I've successfully started rabbitmq with erlang R16B, but again, when I enable rabbithub, rabbitmq stops working. this is the log with the error at startup (after enabling rabbithub).

=INFO REPORT==== 9-Apr-2013::11:41:53 ===
Error description:
undef

Log files (may contain more information):
/var/log/rabbitmq/rabbit@ubuntu.log
/var/log/rabbitmq/rabbit@ubuntu-sasl.log

Stack trace:
[{rabbit_mochiweb,register_context_handler,
["rabbithub",[],#Fun,
"RabbitHub"],
[]},
{rabbit,'-run_boot_step/1-lc$^1/1-1-',1,[]},
{rabbit,run_boot_step,1,[]},
{rabbit,'-start/2-lc$^0/1-0-',1,[]},
{rabbit,start,2,[]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,274}]}]

=INFO REPORT==== 9-Apr-2013::11:41:54 ===
stopped TCP Listener on [::]:5672

=INFO REPORT==== 9-Apr-2013::11:41:54 ===
application: rabbit
exited: {bad_return,{{rabbit,start,[normal,[]]},
{'EXIT',{rabbit,failure_during_boot,undef}}}}
type: temporary

@tonyg
Owner

Right, this is the failure I expected. Certain modules have been renamed recent(ish)ly I think. Some code changes in Rabbithub will be required. They will be minor (I hope) but they are necessary.

@sonoman

Help me understand and I will give a try (though I know nothing about erlang :) )...
This "application_master" that doesn't exist anymore, is a module from rabbit_mochiweb that rabbithub requires ?

@tonyg
Owner

It's actually the other end of the stack trace that's the innermost: rabbit_mochiweb:register_context_handler/4 seems to no longer be present.

@brc859844
Collaborator
@sonoman

actually I'm trying to execute it in our rabbitmq environment....I think I'll try to finish with the re-compilation...running into some issues now (like for example, i can't find the xmerl.hrl include file anywhere in my system :( )

@narbit

Hi sonoman, have you had any success in your attempts to re-align rabbithub with the latest rabbitmq distribution? I'm using rabbitmq-server-3.1.4 and running into the same problem with mochiweb at the moment :(

@brc859844
Collaborator
@narbit

Brett, I picked up a binary you dropped on https://region-a.geo1.objects.hpcloudsvc.com:443/v1/66395568832599/RabbitHub/rabbithub-3.1.0.ez. Is this compiled from the source on https://github.com/brc859844/rabbithub? I was able to enable rabbithub-3.1.0.ez with 3.1.4 but immediately after I restarted RabbitMQ service it terminated with "RabbitMQ: Erlang machine stopped instantly (distribution name conflict?). The service is not restarted as OnFail is set to ignore."

When I disable rabbithub, RabbitMQ service is all happy and running again. Not sure I understand the problem but here is log entries:

=INFO REPORT==== 15-Aug-2013::11:01:02 ===Error description:

{noproc,{gen_server,call,
[rabbit_web_dispatch_registry,
{add,"rabbithub",
[{port,15670}],
#Fun,
#Fun,
{[],"RabbitHub"}},
infinity]}}
Log files (may contain more information):.....

Stack trace:
[{gen_server,call,3,[{file,"gen_server.erl"},{line,188}]},
{rabbit_web_dispatch,register_context_handler,5,[]},
{rabbit,'-run_boot_step/1-lc$^1/1-1-',1,[]},
{rabbit,run_boot_step,1,[]},
{rabbit,'-start/2-lc$^0/1-0-',1,[]},
{rabbit,start,2,[]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,269}]}]

=INFO REPORT==== 15-Aug-2013::11:01:03 ===stopped TCP Listener on 0.0.0.0:5672
=INFO REPORT==== 15-Aug-2013::11:01:03 ===stopped TCP Listener on [::]:5672
=INFO REPORT==== 15-Aug-2013::11:01:03 ===Error description:
{could_not_start,rabbit,
{bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{rabbit,failure_during_boot,
{noproc,
{gen_server,call,
[rabbit_web_dispatch_registry,
{add,"rabbithub",
[{port,15670}],
#Fun,
#Fun,
{[],"RabbitHub"}},
infinity]}}}}}}}

=CRASH REPORT==== 15-Aug-2013::11:01:03 ===
crasher:
initial call: application_master:init/4
pid: <0.148.0>
registered_name: []
exception exit: {bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{rabbit,failure_during_boot,
{noproc,
{gen_server,call,
[rabbit_web_dispatch_registry,
{add,"rabbithub",
[{port,15670}],
#Fun,
#Fun,
{[],"RabbitHub"}},
infinity]}}}}}}
in function application_master:init/4 (application_master.erl, line 133)
ancestors: [<0.147.0>]
messages: [{'EXIT',<0.149.0>,normal}]
links: [<0.147.0>,<0.7.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 2586
stack_size: 27
reductions: 289
neighbours:


Tried downgrading to RabbitMQ 3.1.0, but still ran into similar (same?) problem when enabling plugin:

=CRASH REPORT==== 15-Aug-2013::14:04:46 ===
crasher:
initial call: application_master:init/4
pid: <0.148.0>
registered_name: []
exception exit: {bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{rabbit,failure_during_boot,
{noproc,
{gen_server,call,
[rabbit_web_dispatch_registry,
{add,"rabbithub",
[{port,15670}],
#Fun,
#Fun,
{[],"RabbitHub"}},
infinity]}}}}}}
in function application_master:init/4 (application_master.erl, line 133)
ancestors: [<0.147.0>]
messages: [{'EXIT',<0.149.0>,normal}]
links: [<0.147.0>,<0.7.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 2586
stack_size: 27
reductions: 288
neighbours:

=CRASH REPORT==== 15-Aug-2013::14:05:04 ===
crasher:
initial call: application_master:init/4
pid: <0.148.0>
registered_name: []
exception exit: {bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{rabbit,failure_during_boot,
{noproc,
{gen_server,call,
[rabbit_web_dispatch_registry,
{add,"rabbithub",
[{port,15670}],
#Fun,
#Fun,
{[],"RabbitHub"}},
infinity]}}}}}}
in function application_master:init/4 (application_master.erl, line 133)
ancestors: [<0.147.0>]
messages: [{'EXIT',<0.149.0>,normal}]
links: [<0.147.0>,<0.7.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 2586
stack_size: 27
reductions: 288
neighbours:

I should have mentioned that the instance running on Windows Server 2012.

Natalya

@brc859844
Collaborator
@narbit

Awesome! I noticed 3.1.5 was released in the last couple of days, will definitely want it if it works with rabbithub. Please please let me know what you find as far as the cause of a crash.

Thank you!
Natalya

@brc859844
Collaborator
@narbit

Brett, thank you for tweaking the plugin! I feel I'm so close but no luck :( I disabled rabbithub-3.1.0.ez and uninstalled rabbitmq 3.1.0. I then installed 3.1.5 and enabled rabbithub-3.1.5.ez. But got the same error again:

=CRASH REPORT==== 19-Aug-2013::11:22:31 ===
crasher:
initial call: application_master:init/4
pid: <0.148.0>
registered_name: []
exception exit: {bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{rabbit,failure_during_boot,
{noproc,
{gen_server,call,
[rabbit_web_dispatch_registry,
{add,"rabbithub",
[{port,15670}],
#Fun,
#Fun,
{[],"RabbitHub"}},
infinity]}}}}}}
in function application_master:init/4 (application_master.erl, line 133)
ancestors: [<0.147.0>]
messages: [{'EXIT',<0.149.0>,normal}]
links: [<0.147.0>,<0.7.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 2586
stack_size: 27
reductions: 294

I am not sure how to debug this. But I thought it might be related to a port number since rabbitmq_management was configured to use 15672 and rabbithub was trying to run on 15670. So I changed the config and put rabbitmq_management on 15670, restarted and confirmed that the mgmt web console ran on 15670, all looked good. Then enabled rabbithub and it crashed the service again with the error that looked identical to the above. I've never used Erlang and never tried the plugin development for RabbitMQ. Would I need to install the dev tools and run the source code on the machine where it crashes to get to the bottom of this? Dunno if it matters but it's a 64-bit Windows Server 2012 machine.

@brc859844
Collaborator
@narbit

Heh, otp_win64_R16B01.exe :)

That's great news - the Erlang version dependency you found! The worst case, I can try to downgrade :)

@brc859844
Collaborator
@narbit

Yay! It's definitely much happier now and no errors on startup :) Thank you so much for tweaking the code! As far as implementation goes, there is no support for hub.secret, correct? Also, the rabbithub doc notes that the exchange type can be specified with a query string param, is it also possible to mark queue/exchange/messages as durable using query string? Thank you, Brett, this is great stuff!

@brc859844
Collaborator
@narbit

hub.secret is one of those nice-to-have things but I was hoping the persistence was already there since RabbitMQ supports it. Of course it may not be easy to leverage it from the plugin side, I just don't know. Will wait for the updates and try to learn Erlang meantime :) Thanks, Brett!

@narbit

Brett, it shouldn't be a problem if one or both of the URLs (pub and sub) are HTTPS? For whatever reason everything works for me as long as I don't use SSL. But I see no attempts to post to a HTTPS URL when publishing from HTTP URL. Although I always get get "Accepted" 202 response from the hub in all cases. Then nothing happens and nothing in IIS or RabbitMQ logs (not sure if rabbit logs would even show anything). Just curious.

@brc859844
Collaborator
@narbit

Ah, I see now. The SSL question was more out of curiosity than anything else as a result of a test I ran. I think that for purely RabbitHub purposes hub.secret should do just fine without an overhead of maintaining SSL certificates. I was contemplating including the hmac digest of the entire content into the request body in the absence of hub.secret implementation. Passing the key from sub to pub would become a somewhat manual process. But I may re-think all that. On a different subject, durable queues/messages sound fantastic! :)

UPDATE: just noticed when I create a queue via RabbitHub url, it gets "durable" set to True when I inspect it in web management console. Is this reflective of the actual setting? Also, it doesn't automatically mean that messages are durable, correct?

@narbit

Brett, I was wondering if you had any luck configuring durable queues/messages through RabbitHub? I noticed that the queues created using hub URLs were marked durable in the management console. So after creating two queues I wanted to see what happens if I restart the service. One of them had one unprocessed message in it but no subscribers. Another one has been passing messages to a subscriber. I couldn't start the service after stopping it. It seemed to have trouble with the queue that had a subscriber:

=CRASH REPORT==== 27-Aug-2013::13:42:51 ===
crasher:
initial call: application_master:init/4
pid: <0.304.0>
registered_name: []
exception exit: {bad_return,
{{rabbithub,start,[normal,[]]},
{'EXIT',
{noproc,
{gen_server,call,
[rabbithub_subscription_sup,
{start_child,
[{rabbithub_lease,
{rabbithub_subscription,
{resource,<<"/">>,queue,<<"Group3">>},
"Group3",
"http://localhost/callback.ashx"},
1692656779662140}]},
infinity]}}}}}
in function application_master:init/4 (application_master.erl, line 133)
ancestors: [<0.303.0>]
messages: [{'EXIT',<0.305.0>,normal}]
links: [<0.303.0>,<0.7.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 610
stack_size: 27
reductions: 117
neighbours:

An observation from the log above, the rabbithub_lease in the log above seems to be set to a high value of 1692656779662140 although I passed 315360000 seconds (roughly 10 years) when subscribing. Not sure how that got messed up...

@brc859844
Collaborator
@narbit

Brett, no prob, I figured something else was up :) The subscriber, indeed, was up and running. It currently lives on the same machine. IIS was up and running but it didn't log any communication attempts to callback URL beyond Rabbit restart time.

@brc859844
Collaborator
@narbit

Thanks, Brett! Will give it a try. I've been trying to jump start myself on Erlang but have spent a lot of hours just trying to get the environment up. I was told Windows is not a good a fit for it, so I went with Ubuntu 12.04. But turns out it officially supports r14b, while the latest Ubuntu 13.10 supports r15b :( I have been trying to install r16b manually while using backports and work-arounds for RabbitMQ dependencies. So far it has been a major headache. What OS are you on if you don't mind me asking?

Natalya

@brc859844
Collaborator
@narbit

Interesting info. I wasn't sure how swamped you were, so I just wanted to compile RabbitHub and try chasing the re-start bug you've fixed. I wasn't planning for an extensive Erlang development beyond looking at RabbitHub. I posted in rabbitmq group if plugins could be developed/built on win/cygwin and got this response from a rabbitmq guy: "At the time of writing the released plugins cannot be built by Cygwin from within the public umbrella repository due to the reasons you list (filesystem conventions and escript behavior on Win). If you have a choice then opt for a Unix/GNU platform where the existing build infrastructure will work." I tried Ubuntu with manually compiled r16b and ran into versioning problems due to rabbitmq requiring erlang-nox and Ubuntu trying to give me an old version of erlang (r14b) when I used apt-get to install erlang-nox. This problem and a workaround is described in detail here: http://blog.eriksen.com.br/en/how-install-rabbitmq-latest-erlang-release-debian And although this article explains how to work around erlang-nox dependency, there is still dependency on erlang-dev and xsltproc (libxslt) required to build an umbrella. I found and installed erlang-dev for r16b and one of the recent versions of libxslt. The latter turned out to be buggy. This is where it became annoying and I gave up late Sun night. If I knew things would have worked on Windows with cygwin I would have definitely chosen that path... I may try it sometime this week. Thank you once again, Brett!

@brc859844
Collaborator
@narbit

Brett, the fix worked great, thank you!

As far as infrastructure, here is my question and the answers I got: http://rabbitmq.1065348.n5.nabble.com/plugin-development-on-Windows-possible-td29164.html To be fair it was noted that I could work around with my own build infrastructure. But that's not much info for a newbie :) The rebar friendly fork of rabbit_common you are using is this - https://github.com/jbrisbin/rabbit_common? I'll give it a shot.

Also as far as the code goes, to mark messages persistent would it be correct to modify rabbithub_web.erl, line 504 inside of extract_message(ExchangeResource, ParsedQuery, Req)

from

rabbit_basic:message(ExchangeResource,
list_to_binary(RoutingKey),
[{'content_type', ContentTypeBin}],
Body).
to
rabbit_basic:message(ExchangeResource,
list_to_binary(RoutingKey),
[{'content_type', ContentTypeBin}, {'delivery_mode', 2}],
Body).

???

And lastly, looks like I missed my chance to attend Erlang Factory in 2013, this https://www.erlang-solutions.com/event/erlang-factory-sf-bay-area-2013 would have been closer to home :)

@brc859844
Collaborator
@narbit

Cool! I'll start with brute force hard-coding of delivery mode and if I feel brave enough will see about using query string which would be much nicer. Hmac is another thing I'd like to have for my project. That's if I ever get things in order and running for building :-D

I saw your talk from last year's SFO Erlang factory and ppt on "The Polyglot Rabbit", very cool stuff. I very much agree that the ability to use Rabbit to publish using one protocol and consume using another is underrated. Also, as far as RabbitHub is concerned, no other PubSubHubbub implementation easily supports publishing format-independent (non-Atom, non-RSS) messages. This was big for me. Right now I'm pushing JSON and very happy about it!

@narbit

Followed your instructions for win/cygwin and couldn't believe how easy it was compared to official instructions and looking back at my struggles with Ubuntu. Made my change, compiled and will deploy and test tomorrow. Thank you so much for pointing me in the right direction!

@brc859844
Collaborator
@narbit

Yeah, I'm so excited it finally worked :) I'm going to see about passing delivery mode in query string and, yeah, SSL support sounds terrific too!

@narbit

Brett, I noticed rabbithub uses error_logger to log errors/info messages. Do you get any rabbithub specific output from these statements? For example, register_subscription_pid in rabbithub_subscription.erl calls error_logger:info_report({startup, Result, ProcessModule, Lease}) but I see no INFO REPORT in the rabbitmq log file. I tried calling error_logger:info_report from within a function in rabbithub_web.erl but that didn't get logged either. Almost like this statement gets skipped.

@brc859844
Collaborator
@narbit

Got it. I thought error_logger was supposed to just work. I looked at some other rabbit plugins and all of them use error_logger almost exclusively. So I thought I was missing a reference somewhere or something like that. But rabbit_log worked just great, thank you! :)

@brc859844
Collaborator
@narbit

Agreed! And can totally relate to being inherently lazy :)

@narbit

Brett, another question. I was trying to figure out what happens to the queue/messages if subscriber, the owner of the callback URL, goes offline and becomes unreachable. From looking at the code my understanding is that if rabbithub_consumer is unable to deliver a message to a callback URL, it will throw an error and then delete a subscription? If the queue is durable, the messages will then pile up there but will never be consumed unless subscriber re-subscribes? If subscriber comes back online and re-subscribes, will queued messages then be delivered?

@brc859844
Collaborator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.