Fix rabbitmq #57

drnic · 2013-08-16T00:28:52Z

Probably needs same upgrade as other services did:

# cf create-service
1: postgresql 9.2
2: rabbitmq 3.0
3: redis 2.6
What kind?> 2

Name?> rabbitmq-4cea0

1: default: Dedicated server, shared VM, 250MB storage, 10 connections
Which plan?> 1

Creating service rabbitmq-4cea0... FAILED
CFoundry::ServerError: 10001: Server error
cat ~/.cf/crash # for more details

The text was updated successfully, but these errors were encountered:

drnic · 2013-08-16T04:20:39Z

Issue is in the node:

==> /var/vcap/sys/log/rabbit_node/rabbit_node.log <==
[2013-08-16 04:19:52.592027] rabbit_node_default_0 - pid=23275 tid=b41e fid=6dec  DEBUG -- RMQaaS-Node: Provision request: {"plan":"default","version":"3.0"} from _INBOX.79852ae58953fd1771a6fee18d
[2013-08-16 04:19:52.592609] rabbit_node_default_0 - pid=23275 tid=b41e fid=6dec  DEBUG -- RMQaaS-Node: Provision Request Details - plan=default, credentials=, version=3.0
[2013-08-16 04:19:53.848273] rabbit_node_default_0 - pid=23275 tid=92ea fid=51fa  DEBUG -- RMQaaS-Node: Sending announcement for everyone
[2013-08-16 04:19:58.161711] rabbit_node_default_0 - pid=23275 tid=b41e fid=6dec  ERROR -- Error provision instance: Error Code: 30503, Error Message: Service start timeout
[2013-08-16 04:19:58.349281] rabbit_node_default_0 - pid=23275 tid=b41e fid=6dec   WARN -- Exception at on_provision Error Code: 30503, Error Message: Service start timeout

drnic · 2013-08-16T04:23:10Z

Looks like its using bad rootfs location:

==> /var/vcap/sys/log/monit/warden_ctl.err.log <==
tar: /var/vcap/stemcell_base.tar.gz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Child returned status 2
tar: Exiting with failure status due to previous errors

drnic · 2013-08-16T04:34:12Z

Looks like the last error is common to redis_node_ng (and probably others). Created #58

rkoster · 2013-08-16T13:37:36Z

Don't think the above is the issue found the following in

fid=c1c4 container/linux.rb/do_create:104 handle=17416mb2jok   DEBUG -- Container created
fid=c1c4 container/linux.rb/do_create:110 handle=17416mb2jok   DEBUG -- Container started
fid=c1c4 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000484
fid=c1c4 container/base.rb/dispatch:318 handle=17416mb2jok,request={"bind_mounts"=>["#<Warden::Protocol::CreateRequest::BindMount:0x0
00000014cbd40>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014cb2a0>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014ca8a0>", "#<Warden::Protocol::CreateRequest::BindMount:0x0000000
14c9e28>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014d13a8>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014d09a8>"]},response={"handle"=>"17416mb2jok"}    INFO -- create (took 0
.378770)
fid=2cb1 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000354
fid=2cb1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"chown -R vcap:vcap /var/vcap/
sys/log/monit /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/data/packages/erlang/0.1-dev.1", "privil
eged"=>true},response={"job_id"=>13}    INFO -- spawn (took 0.005973)
fid=8b2f container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000378
fid=2cb1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "job_id"=>13},response={"exit_status"=>0
, "stdout"=>"", "stderr"=>""}    INFO -- link (took 0.040395)
fid=2cb1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"chown -R vcap:vcap /var/vcap/
sys/log/monit /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/data/packages/erlang/0.1-dev.1", "privil
eged"=>true},response={"exit_status"=>0, "stdout"=>"", "stderr"=>""}    INFO -- run (took 0.047099)
fid=c8e0 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000357
fid=c8e0 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "rate"=>94371, "burst"=>94371},response=
{"rate"=>94371, "burst"=>94371}    INFO -- limit_bandwidth (took 0.013958)
fid=815d container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000396
fid=815d container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"/var/vcap/store/rabbitmq_comm
on/bin/warden_service_ctl start /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/store/rabbitmq_common
/var/vcap/packages/rabbitmq-3.0 /var/vcap/data/packages/erlang/0.1-dev.1 a4c9a83e-082a-435c-80f1-783916040be9"},response={"job_id"=>14}    INFO -- spawn (took 0.004846)
fid=0207 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000262
fid=0207 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "host_port"=>15005, "container_port"=>10
001},response={"host_port"=>15005, "container_port"=>10001}    INFO -- net_in (took 0.014269)
fid=8b2f container/spawn.rb/set_deferred_success:134 handle=17416mb2jok,stdout=,stderr=    WARN -- Exited with status 1 (0.026s): [["
/var/vcap/data/packages/rabbit_node_ng/0.1-dev.1/warden/src/closefds/closefds", "/var/vcap/data/packages/rabbit_node_ng/0.1-dev.1/warden/src/closefds/closefds"], "/var/vcap/store/rabbit/containers/17416mb
2jok/bin/iomux-link", "-w", "/var/vcap/store/rabbit/containers/17416mb2jok/jobs/14/cursors", "/var/vcap/store/rabbit/containers/17416mb2jok/jobs/14"]
fid=8b2f container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000366
fid=2ca8 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={"state"=>"active", "events"=>[
], "host_ip"=>"10.254.0.17", "container_ip"=>"10.254.0.18", "container_path"=>"/var/vcap/store/rabbit/containers/17416mb2jok", "memory_stat"=>"#<Warden::Protocol::InfoResponse::MemoryStat:0x0000000100bfd0
>", "cpu_stat"=>"#<Warden::Protocol::InfoResponse::CpuStat:0x000000010107d8>", "bandwidth_stat"=>"#<Warden::Protocol::InfoResponse::BandwidthStat:0x00000001991910>", "job_ids"=>[14]}    INFO -- info (took
 0.016397)
fid=489b container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={"state"=>"active", "events"=>[
], "host_ip"=>"10.254.0.17", "container_ip"=>"10.254.0.18", "container_path"=>"/var/vcap/store/rabbit/containers/17416mb2jok", "memory_stat"=>"#<Warden::Protocol::InfoResponse::MemoryStat:0x0000000143dcc0
>", "cpu_stat"=>"#<Warden::Protocol::InfoResponse::CpuStat:0x00000001525868>", "bandwidth_stat"=>"#<Warden::Protocol::InfoResponse::BandwidthStat:0x0000000167d210>", "job_ids"=>[]}    INFO -- info (took 0
.025080)
fid=2f04 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000403
fid=2f04 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"/var/vcap/store/rabbitmq_comm
on/bin/warden_service_ctl stop /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/store/rabbitmq_common"}
,response={"job_id"=>15}    INFO -- spawn (took 0.005861)
fid=8b2f container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000427
fid=2f04 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "job_id"=>15},response={"exit_status"=>0
, "stdout"=>"", "stderr"=>""}    INFO -- link (took 0.015982)
fid=2f04 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"/var/vcap/store/rabbitmq_comm
on/bin/warden_service_ctl stop /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/store/rabbitmq_common"}
,response={"exit_status"=>0, "stdout"=>"", "stderr"=>""}    INFO -- run (took 0.022702)
fid=1aa1 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000377
fid=1aa1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={}    INFO -- stop (took 0.0220
32)
fid=21fa container/linux.rb/do_destroy:129 handle=17416mb2jok   DEBUG -- Container destroyed
fid=21fa container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={}    INFO -- destroy (took 0.1
17148)

By using tail -f /var/vcap/sys/log/rabbit/warden/warden.log | steno-prettify (can only be used when first running gem install steno -r)

drnic · 2013-08-16T13:44:53Z

Right, I moved it off into another ticket as a wtf discovery of why that code path exists and can it be removed?

I'm still not sure why RabbitMQ provisioning is failing.

--

Dr Nic Williams
Stark & Wayne LLC - the consultancy for Cloud Foundry
http://starkandwayne.com
+1 415 860 2185
twitter: drnic

On Fri, Aug 16, 2013 at 6:37 AM, Ruben Koster notifications@github.com
wrote:

Don't think the above is the issue found the following in

2013-08-16 13:28:24.964936 Warden::Container::Linux pid=27128 tid=ac21 fid=c1c4 container/linux.rb/do_create:104 handle=17416mb2jok   DEBUG -- Container created
2013-08-16 13:28:25.184764 Warden::Container::Linux pid=27128 tid=ac21 fid=c1c4 container/linux.rb/do_create:110 handle=17416mb2jok   DEBUG -- Container started
2013-08-16 13:28:25.185497 Warden::Container::Linux pid=27128 tid=ac21 fid=c1c4 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000484
2013-08-16 13:28:25.185684 Warden::Container::Linux pid=27128 tid=ac21 fid=c1c4 container/base.rb/dispatch:318 handle=17416mb2jok,request={"bind_mounts"=>["#<Warden::Protocol::CreateRequest::BindMount:0x0
00000014cbd40>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014cb2a0>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014ca8a0>", "#<Warden::Protocol::CreateRequest::BindMount:0x0000000
14c9e28>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014d13a8>", "#<Warden::Protocol::CreateRequest::BindMount:0x000000014d09a8>"]},response={"handle"=>"17416mb2jok"}    INFO -- create (took 0
.378770)
2013-08-16 13:28:25.193724 Warden::Container::Linux pid=27128 tid=ac21 fid=2cb1 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000354
2013-08-16 13:28:25.193905 Warden::Container::Linux pid=27128 tid=ac21 fid=2cb1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"chown -R vcap:vcap /var/vcap/
sys/log/monit /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/data/packages/erlang/0.1-dev.1", "privil
eged"=>true},response={"job_id"=>13}    INFO -- spawn (took 0.005973)
2013-08-16 13:28:25.234291 Warden::Container::Linux pid=27128 tid=ac21 fid=8b2f container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000378
2013-08-16 13:28:25.234522 Warden::Container::Linux pid=27128 tid=ac21 fid=2cb1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "job_id"=>13},response={"exit_status"=>0
, "stdout"=>"", "stderr"=>""}    INFO -- link (took 0.040395)
2013-08-16 13:28:25.234918 Warden::Container::Linux pid=27128 tid=ac21 fid=2cb1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"chown -R vcap:vcap /var/vcap/
sys/log/monit /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/data/packages/erlang/0.1-dev.1", "privil
eged"=>true},response={"exit_status"=>0, "stdout"=>"", "stderr"=>""}    INFO -- run (took 0.047099)
2013-08-16 13:28:25.250089 Warden::Container::Linux pid=27128 tid=ac21 fid=c8e0 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000357
2013-08-16 13:28:25.250212 Warden::Container::Linux pid=27128 tid=ac21 fid=c8e0 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "rate"=>94371, "burst"=>94371},response=
{"rate"=>94371, "burst"=>94371}    INFO -- limit_bandwidth (took 0.013958)
2013-08-16 13:28:25.256461 Warden::Container::Linux pid=27128 tid=ac21 fid=815d container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000396
2013-08-16 13:28:25.256645 Warden::Container::Linux pid=27128 tid=ac21 fid=815d container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"/var/vcap/store/rabbitmq_comm
on/bin/warden_service_ctl start /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/store/rabbitmq_common
/var/vcap/packages/rabbitmq-3.0 /var/vcap/data/packages/erlang/0.1-dev.1 a4c9a83e-082a-435c-80f1-783916040be9"},response={"job_id"=>14}    INFO -- spawn (took 0.004846)
2013-08-16 13:28:25.275011 Warden::Container::Linux pid=27128 tid=ac21 fid=0207 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000262
2013-08-16 13:28:25.275134 Warden::Container::Linux pid=27128 tid=ac21 fid=0207 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "host_port"=>15005, "container_port"=>10
001},response={"host_port"=>15005, "container_port"=>10001}    INFO -- net_in (took 0.014269)
2013-08-16 13:28:25.280344 Warden::Container::Linux pid=27128 tid=ac21 fid=8b2f container/spawn.rb/set_deferred_success:134 handle=17416mb2jok,stdout=,stderr=    WARN -- Exited with status 1 (0.026s): [["
/var/vcap/data/packages/rabbit_node_ng/0.1-dev.1/warden/src/closefds/closefds", "/var/vcap/data/packages/rabbit_node_ng/0.1-dev.1/warden/src/closefds/closefds"], "/var/vcap/store/rabbit/containers/17416mb
2jok/bin/iomux-link", "-w", "/var/vcap/store/rabbit/containers/17416mb2jok/jobs/14/cursors", "/var/vcap/store/rabbit/containers/17416mb2jok/jobs/14"]
2013-08-16 13:28:25.280885 Warden::Container::Linux pid=27128 tid=ac21 fid=8b2f container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000366
2013-08-16 13:28:25.292477 Warden::Container::Linux pid=27128 tid=ac21 fid=2ca8 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={"state"=>"active", "events"=>[
], "host_ip"=>"10.254.0.17", "container_ip"=>"10.254.0.18", "container_path"=>"/var/vcap/store/rabbit/containers/17416mb2jok", "memory_stat"=>"#<Warden::Protocol::InfoResponse::MemoryStat:0x0000000100bfd0
>", "cpu_stat"=>"#<Warden::Protocol::InfoResponse::CpuStat:0x000000010107d8>", "bandwidth_stat"=>"#<Warden::Protocol::InfoResponse::BandwidthStat:0x00000001991910>", "job_ids"=>[14]}    INFO -- info (took
 0.016397)
2013-08-16 13:28:30.375134 Warden::Container::Linux pid=27128 tid=ac21 fid=489b container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={"state"=>"active", "events"=>[
], "host_ip"=>"10.254.0.17", "container_ip"=>"10.254.0.18", "container_path"=>"/var/vcap/store/rabbit/containers/17416mb2jok", "memory_stat"=>"#<Warden::Protocol::InfoResponse::MemoryStat:0x0000000143dcc0
>", "cpu_stat"=>"#<Warden::Protocol::InfoResponse::CpuStat:0x00000001525868>", "bandwidth_stat"=>"#<Warden::Protocol::InfoResponse::BandwidthStat:0x0000000167d210>", "job_ids"=>[]}    INFO -- info (took 0
.025080)
2013-08-16 13:28:30.384576 Warden::Container::Linux pid=27128 tid=ac21 fid=2f04 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000403
2013-08-16 13:28:30.384791 Warden::Container::Linux pid=27128 tid=ac21 fid=2f04 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"/var/vcap/store/rabbitmq_comm
on/bin/warden_service_ctl stop /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/store/rabbitmq_common"}
,response={"job_id"=>15}    INFO -- spawn (took 0.005861)
2013-08-16 13:28:30.400686 Warden::Container::Linux pid=27128 tid=ac21 fid=8b2f container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000427
2013-08-16 13:28:30.401001 Warden::Container::Linux pid=27128 tid=ac21 fid=2f04 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "job_id"=>15},response={"exit_status"=>0
, "stdout"=>"", "stderr"=>""}    INFO -- link (took 0.015982)
2013-08-16 13:28:30.401557 Warden::Container::Linux pid=27128 tid=ac21 fid=2f04 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok", "script"=>"/var/vcap/store/rabbitmq_comm
on/bin/warden_service_ctl stop /var/vcap/store/rabbit/instances/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/sys/service-log/rabbit/a4c9a83e-082a-435c-80f1-783916040be9 /var/vcap/store/rabbitmq_common"}
,response={"exit_status"=>0, "stdout"=>"", "stderr"=>""}    INFO -- run (took 0.022702)
2013-08-16 13:28:30.424860 Warden::Container::Linux pid=27128 tid=ac21 fid=1aa1 container/base.rb/write_snapshot:360 handle=17416mb2jok   DEBUG -- Wrote snapshot in 0.000377
2013-08-16 13:28:30.425029 Warden::Container::Linux pid=27128 tid=ac21 fid=1aa1 container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={}    INFO -- stop (took 0.0220
32)
2013-08-16 13:28:30.543009 Warden::Container::Linux pid=27128 tid=ac21 fid=21fa container/linux.rb/do_destroy:129 handle=17416mb2jok   DEBUG -- Container destroyed
2013-08-16 13:28:30.543368 Warden::Container::Linux pid=27128 tid=ac21 fid=21fa container/base.rb/dispatch:318 handle=17416mb2jok,request={"handle"=>"17416mb2jok"},response={}    INFO -- destroy (took 0.1
17148)

By using `tail -f /var/vcap/sys/log/rabbit/warden/warden.log | steno-prettify` (can only be used when first running `gem install steno -r`)

Reply to this email directly or view it on GitHub:
#57 (comment)

rkoster · 2013-08-16T13:51:35Z

Running tail -f /var/vcap/sys/service-log/rabbit/**/*.log after running cf create-service --name rabbit-test --plan default --offering rabbitmq gives:

==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/rabbitmq_stderr.log <==
exec: 28: /var/vcap/packages/erlang/lib/erlang/erts-5.8.2/bin/erlexec: not found

==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/rabbitmq_stdout.log <==

==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/warden_service_ctl.err.log <==

==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/warden_service_ctl.log <==
Process 18 is not running

drnic · 2013-08-16T14:01:29Z

service-log ?!? I never noticed logs were going to a different folder. Damn people not sticking to conventions :)

--

Dr Nic Williams
Stark & Wayne LLC - the consultancy for Cloud Foundry
http://starkandwayne.com
+1 415 860 2185
twitter: drnic

On Fri, Aug 16, 2013 at 6:51 AM, Ruben Koster notifications@github.com
wrote:

Running tail -f /var/vcap/sys/service-log/rabbit/**/*.log after running cf create-service --name rabbit-test --plan default --offering rabbitmq gives:
==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/rabbitmq_stderr.log <==
exec: 28: /var/vcap/packages/erlang/lib/erlang/erts-5.8.2/bin/erlexec: not found
==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/rabbitmq_stdout.log <==
==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/warden_service_ctl.err.log <==
==> acc7a135-befd-4ec3-a9cb-bee7d168f12b/warden_service_ctl.log <==
Process 18 is not running
Reply to this email directly or view it on GitHub:
#57 (comment)

drnic · 2013-08-16T14:39:34Z

Ok, so /var/vcap/sys/log is for the node itself; and /var/vcap/sys/service-log/ is for the logs of the user service instances. Need to update my tail helper.

rkoster · 2013-08-16T14:39:53Z

can not find erlang in list of installed packages for rootfs

drnic · 2013-08-16T14:44:16Z

Yeah, I guess when they moved from the stemcell-built rootfs to the new system some packages didn't make it across.

drnic mentioned this issue Aug 16, 2013

Add erlang package to allow rabbitmq to run within warden container cloudfoundry/cflinuxfs2#1

Closed

drnic closed this as completed in 0234f51 Aug 16, 2013

trastle mentioned this issue Oct 21, 2013

[rabbitmq] "cf create-service" fails with timeout #93

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix rabbitmq #57

Fix rabbitmq #57

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

rkoster commented Aug 16, 2013

drnic commented Aug 16, 2013

By using `tail -f /var/vcap/sys/log/rabbit/warden/warden.log | steno-prettify` (can only be used when first running `gem install steno -r`)

rkoster commented Aug 16, 2013

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

rkoster commented Aug 16, 2013

drnic commented Aug 16, 2013

Fix rabbitmq #57

Fix rabbitmq #57

Comments

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

rkoster commented Aug 16, 2013

drnic commented Aug 16, 2013

By using tail -f /var/vcap/sys/log/rabbit/warden/warden.log | steno-prettify (can only be used when first running gem install steno -r)

rkoster commented Aug 16, 2013

drnic commented Aug 16, 2013

drnic commented Aug 16, 2013

rkoster commented Aug 16, 2013

drnic commented Aug 16, 2013

By using `tail -f /var/vcap/sys/log/rabbit/warden/warden.log | steno-prettify` (can only be used when first running `gem install steno -r`)