FailoverRoute issue

[0xd3adsh3ll]

Hello again.
Currently I have config like this http://pastebin.com/Wfn08B5F
I have 1 server with mcrouter and 2 servers with memcached
For 'delete' - type AllAsyncRoute
For 'get' - type FailoverRoute
For 'set' - type AllAsyncRoute

From docs I read this:
"FailoverRoute

Sends the request to the first child in the list and waits for the reply. If the reply is a non-error, returns it immediately. Otherwise, sends the request to the second child, and so on. If all children respond with errors, returns the last error reply."

When I'm start my java app and run 1000 request on !set! to mcrouter server and test "AllAsyncRoute" (disconnect memcached1 or memcached2 via iptables rule) - it's OK, i can set all 1000 requests to both memcached servers, but when I start java app and run 1000 request on !get! from mcrouter then I get only keys from 1 of 2 servers and nothing all keys. I don't see failover mode.
Sorry for my stupid question, can you help me please?
Thanks.

[0xd3adsh3ll]

SET:

1. Run java app (set mode) steps from 1 to 1000
2. Create iptables rules for blocking memcached2 - ok, mcrouter "set" to memcached1 - no errors
3. Delete iptables rules for memcached2 - ok, mcrouter delete "TKO" from memcached2 and "set" to all nodes - no errors
4. Create iptables rules for blocking memcached1 - ok, mcrouter "set" to memcached2 - no errors
5. Delete iptables rules for memcached1 - ok, mcrouter delete "TKO" from memcached1 and "set" to all nodes - no errors

Java app stop, all 1000 requests a set, no errors

GET:

1. Run java app (get mode) steps from 1 to 1000
2. Java app connect to mcrouter and do get keys and values from 1 to 1000
3. Java app stop all 1000 requests and create stats file with info.
4. I'm viewing log file from memcached1 and see 1000 get requests
5. I'm viewing log file from memcached2 and don't see any requests. But "get" type - FailoverRoute

[pavlo-fb]

Hi,
As I understand, you do not disable memcached1 while sending GET requests?
If the first server was available and didn't return an error reply, FailoverRoute will not send the request to second server. That's why you see all 1000 requests processed by memcached1. FailoverRoute will send to memcached2 only when it receives an error reply (miss counts for success) from memcached1.
If you want to send a request to memcached2 on misses (i.e. when key is not found on memcached1), use MissFailoverRoute instead of FailoverRoute.

[0xd3adsh3ll]

Hello,
Thanks you very much! MissFailoverRoute - very well for me.
pavlo-fb, can you provide for me all documentation about mcrouter? Docs from github is full?
We starts high load project and mcrouter is amazing and the best solution for us.

[pavlo-fb]

Hi Maxim,
Yes, mcrouter wiki is the most comprehensive and up to date documentation for mcrouter.
Thanks for your feedback!

[alikhtarov]

@0xd3adsh3ll and if you don't find an answer on the wiki, please open an issue here and we'll improve our docs.

[0xd3adsh3ll]

Hello,
Ok, thanks!

[liyufu]

@0xd3adsh3ll
http://pastebin.com/Wfn08B5F

your config file can't opened anywhere , i have some problem as you writed,now my config is not run the AllAsyncRoute ,please help me for you config, thanks!.

addtion:there is my issue,
#33

[0xd3adsh3ll]

@liyufu
my config: http://pastebin.com/E1tVfAH0
This config works fine on production about 1 month.

[liyufu]

@0xd3adsh3ll
Thanks , you almost never used the FailoverRoute ?

if memcached1 is downed?the mcrouter can get data from memcached2?

after memcached1 is resolved and started, how to solve the memcached2 is empty problem?

thanks

[0xd3adsh3ll]

Hello @liyufu,
I'm use:
for delete - AllSyncRoute - works fine with my app, AllAsyncRoute works fine too, but it's two different methods.
for set - MissFailoverRoute, if memcached1 marked as TKO (down, unreachable, etc) then request send to memcached2, if memcached2 can't response mcrouter return error.
for get - AllFastestRoute, get from fastest node (memcached1 or memcached2)

I'm don't use FailoverRoute because: "If the first server was available and didn't return an error reply, FailoverRoute will not send the request to second server."
If you have more questions please see facebook group https://www.facebook.com/groups/mcrouter/ or create new issue.

[liyufu]

Thanks.