Ordering of rewrites and matches in relay.conf

[mhollick]

Morning,
It looks like rewrites are performed before matches, such as a match placed before a rewrite will be sent data which has been subject to the rewrite.

From the documentation:

match * send to old;

rewrite ... ;

match * send to new;

I understand that this is not the intended behaviour.

Methodology:

relay invocation in terminal session 1:

$ /usr/bin/relay -p 4000 -w 2 -b 2500 -q 25000 -H maxwell -f ./relay.test.conf
[2014-09-11 11:07:19] starting carbon-c-relay v0.32 (2014-09-10)
configuration:
    relay hostname = maxwell
    listen port = 4000
    workers = 2
    send batch size = 2500
    server queue size = 25000
    routes configuration = ./relay.test.conf

parsed configuration follows:
cluster new
    forward
        127.0.0.1:5000
    ;
cluster old
    forward
        127.0.0.1:6000
    ;

match ^carbon\.relays\..*$
    send to blackhole
    stop
    ;
match *
    send to old
    ;
rewrite ^foo\.(.*)
    into bar.\1
    ;
match *
    send to new
    ;

listening on tcp4 0.0.0.0 port 4000
listening on UNIX socket /tmp/.s.carbon-c-relay.4000
starting 2 workers
starting statistics collector

Listener for cluster old in terminal 2:

nc -kl 5000

Listener for cluster new in terminal 3:

nc -kl 6000

Feed data into graphite from terminal 4:

echo "foo.monkey 4 `date +%s`" | nc 127.0.0.1 4000

Expected result in cluster old:

foo.monkey 4 1410433961

Expected result in cluster new:

bar.monkey 4 1410433961

Actual result in cluster old:

bar.monkey 4 1410433961

Actual result in cluster new:

bar.monkey 4 1410433961

[mhollick]

Forgot to say thankyou...
My mother would cry.

[grobian]

Hmmm, this surprises me a lot indeed!

[grobian]

Crap, no good, evaluation problem here. The destinations for a metric are evaluated in one go, thus the rewrite inbetween changes the value for the entire run.

[grobian]

% ./relay -f issue14.conf -p 4000
[2014-09-12 15:21:18] starting carbon-c-relay v0.32 (726485-dirty)
configuration:
    relay hostname = gaia.local
    listen port = 4000
    workers = 16
    send batch size = 2500
    server queue size = 25000
    routes configuration = issue14.conf

parsed configuration follows:
cluster new
    forward
        127.0.0.1:5000
    ;
cluster old
    forward
        127.0.0.1:6000
    ;

match ^carbon\.relays\..*$
    send to blackhole
    stop
    ;
match *
    send to old
    ;
rewrite ^foo\.(.*)
    into bar.\1
    ;
match *
    send to new
    ;

listening on tcp4 0.0.0.0 port 4000
listening on tcp6 :: port 4000
listening on UNIX socket /tmp/.s.carbon-c-relay.4000
starting 16 workers
starting statistics collector

% echo "foo.monkey 4 `date +%s`" | nc 127.0.0.1 4000
% nc -l 5000
bar.monkey 4 1410528100
% nc -l 6000
foo.monkey 4 1410528100

...
ok, getting close, but not quite yet

[grobian]

oi, old is 6000, so it IS correct (thought I got it reversed somehow)

[grobian]

I'd appreciate a test of this commit. I had to do some sneaky malloc changes (see commit message), so it is not impossible the thing crashes after handling some data. I think I traced all starts and ends, so it should be fine from that point of view.

[mhollick]

Thanks,
I shall run it through the wringer.

[mhollick]

Looks Good!
Sorry I got the expected results the wrong way round in the example..