performance statistics only in debug mode?

[mhollick]

Good evening,

It looks like there may be some issue with the routing of the performance statistics when not running with debug enabled:

Running relay with debug enabled

$ relay -f /etc/cc_relay.conf -p 2103 -w 2 -b 2500 -q 25000 -d -H maxwell
[2014-09-02 20:14:19] starting carbon-c-relay v0.31 (2014-09-02)
configuration:
    relay hostname = maxwell
    listen port = 2103
    workers = 2
    send batch size = 2500
    server queue size = 25000
    debug = true
    routes configuration = /etc/cc_relay.conf

parsed configuration follows:
cluster localhost
    forward
        127.0.0.1:2003
    ;

match *
    send to localhost
    ;

listening on tcp4 0.0.0.0 port 2103
listening on UNIX socket /tmp/.s.carbon-c-relay.2103
starting 2 workers
starting statistics collector
[2014-09-02 20:14:38] failed to send() to 127.0.0.1:2003: Broken pipe
[2014-09-02 20:14:41] server 127.0.0.1:2003: OK
carbon.relays.maxwell.dispatcher1.metricsReceived 920 1409685319
carbon.relays.maxwell.dispatcher1.wallTime_us 15045 1409685319
carbon.relays.maxwell.dispatcher2.metricsReceived 1250 1409685319
carbon.relays.maxwell.dispatcher2.wallTime_us 21912 1409685319
carbon.relays.maxwell.metricsReceived 2170 1409685319
carbon.relays.maxwell.dispatch_wallTime_us 36957 1409685319
carbon.relays.maxwell.dispatch_busy 0 1409685319
carbon.relays.maxwell.dispatch_idle 2 1409685319
carbon.relays.maxwell.destinations.127_0_0_1:2003.sent 2170 1409685319
carbon.relays.maxwell.destinations.127_0_0_1:2003.queued 0 1409685319
carbon.relays.maxwell.destinations.127_0_0_1:2003.dropped 0 1409685319
carbon.relays.maxwell.destinations.127_0_0_1:2003.wallTime_us 19995 1409685319
carbon.relays.maxwell.destinations.internal.sent 0 1409685319
carbon.relays.maxwell.destinations.internal.queued 0 1409685319
carbon.relays.maxwell.destinations.internal.dropped 0 1409685319
carbon.relays.maxwell.destinations.internal.wallTime_us 0 1409685319
carbon.relays.maxwell.metricsSent 2170 1409685319
carbon.relays.maxwell.metricsQueued 0 1409685319
carbon.relays.maxwell.metricsDropped 0 1409685319
carbon.relays.maxwell.server_wallTime_us 19995 1409685319
carbon.relays.maxwell.connections 217 1409685319
carbon.relays.maxwell.disconnects 217 1409685319
^Ccaught SIGINT, terminating...
[2014-09-02 20:15:56] shutting down...
[2014-09-02 20:15:56] listener for port 2103 closed
[2014-09-02 20:15:57] collector stopped
[2014-09-02 20:15:57] stopped worker 1 2 3 (2014-09-02 20:15:58)
[2014-09-02 20:15:58] routing stopped

Running relay without debug enabled:

relay -f /etc/cc_relay.conf -p 2103 -w 2 -b 2500 -q 25000 -H maxwell
[2014-09-02 20:19:12] starting carbon-c-relay v0.31 (2014-09-02)
configuration:
    relay hostname = maxwell
    listen port = 2103
    workers = 2
    send batch size = 2500
    server queue size = 25000
    routes configuration = /etc/cc_relay.conf

parsed configuration follows:
cluster localhost
    forward
        127.0.0.1:2003
    ;

match *
    send to localhost
    ;

listening on tcp4 0.0.0.0 port 2103
listening on UNIX socket /tmp/.s.carbon-c-relay.2103
starting 2 workers
starting statistics collector
[2014-09-02 20:20:12] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:12] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:12] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:13] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:13] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:13] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:13] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:14] failed to send() to internal:2103: Socket operation on non-socket
[2014-09-02 20:20:14] failed to send() to internal:2103: Socket operation on non-socket
...........

I also noticed that a UNIX socket has been opened when not running in debug mode

Cheers,

Matthew.

[grobian]

The unix socket is meant for usage by external programs (like the tcp socket), so regardless of debug mode, it is intended to be opened. The send() failure is a regression.

[mhollick]

Hi,
I have compiled and tested the fix - it work great - thanks.
Also, btw, I am using the newish rewrite capabilities and finding them pretty awesome.

Regards,

Matthew.

[grobian]

thanks! I haven't had the chance to extensively try the rewrites yet, so all feedback is welcome :)