Added support for unavailable replica servers #1159

vaseker · 2014-04-10T13:31:11Z

3 fixes for a ping strategy.

Implemented support for unavailable replica servers
No more lowest.runtimeStats.pingMs = undefined
Configurable options for ping timeout/interval.

christkv · 2014-04-10T15:26:38Z

does it pass the test suite ?

make sure mongod is in your path and run

node test/runner.js -t replicaset

vaseker · 2014-04-10T15:48:39Z

No, test fails.
It fails also on 1.4 branch.

Console output below:

node test/runner.js -t replicaset
DEBUG: ** Starting a replica set with 4 nodes
DEBUG: stdout: note: noprealloc may hurt performance in many applications
about to fork child process, waiting until server is ready for connections.
forked process: 5371
all output going to: /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/data/log-30010
child process started successfully, parent exiting

DEBUG: stderr: 
DEBUG: stdout: note: noprealloc may hurt performance in many applications
about to fork child process, waiting until server is ready for connections.
forked process: 5373
all output going to: /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/data/log-30013
child process started successfully, parent exiting

DEBUG: stderr: 
DEBUG: stdout: note: noprealloc may hurt performance in many applications
about to fork child process, waiting until server is ready for connections.
forked process: 5377
all output going to: /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/data/log-30011
child process started successfully, parent exiting

DEBUG: stderr: 
DEBUG: stdout: note: noprealloc may hurt performance in many applications
about to fork child process, waiting until server is ready for connections.
forked process: 5376
all output going to: /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/data/log-30012
child process started successfully, parent exiting

DEBUG: stderr: 
DEBUG: ** Ensuring members are up...
.[ensureUp] - 30010 :: 120
.[ensureUp] - 30010 :: 119
.[ensureUp] - 30010 :: 118
.[ensureUp] - 30010 :: 117
.[ensureUp] - 30010 :: 116
.[ensureUp] - 30010 :: 115
.[ensureUp] - 30010 :: 114
all members up!

STARTING 0 :: 67 :: Should correctly connect to arbiter with single connection
[ensureUp] - 30010 :: 120
[ensureUp] - 30010 :: 120
✔ Should correctly connect to arbiter with single connection
STARTING 0 :: 66 :: Should fail due to w:5 and wtimeout:1 combined with duplicate key errors with unordered batch api

/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/connection/base.js:245
        throw message;      
              ^
AssertionError: null == 0
    at equal (/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/node_modules/integra/lib/test_control.js:37:14)
    at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/test/tests/repl_set/batch_api_tests.js:266:16
    at processLegacyOperations (/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/collection/batch/ordered.js:452:12)
    at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/collection/batch/ordered.js:353:13
    at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/collection/core.js:234:22
    at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/collection/core.js:599:7
    at Server.Base._callHandler (/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/connection/base.js:445:41)
    at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/connection/server.js:468:18
    at MongoReply.parseBody (/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
    at null.<anonymous> (/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/connection/server.js:426:20)

christkv · 2014-04-10T15:51:36Z

what server version are you running against ? it's been tested against 2.4.X and 2.6.X

vaseker · 2014-04-10T15:52:29Z

db version v2.4.9

vaseker · 2014-04-10T16:09:42Z

Updated to 2.6.
Test fails at

STARTING 0 :: 56 :: Should Correctly Vary read server when using readpreference NEAREST

/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/db.js:297
          throw err;
                ^
TypeError: Cannot read property 'port' of undefined
    at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/test/tests/repl_set/read_preferences_spec_tests.js:186:40
    at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/db.js:294:11
    at process._tickCallback (node.js:415:13)

I'll try to resolve this

christkv · 2014-04-10T16:10:20Z

I'm checking it aswell right now.

On Thu, Apr 10, 2014 at 6:09 PM, Dimka notifications@github.com wrote:

Updated to 2.6.
Test fails at

STARTING 0 :: 56 :: Should Correctly Vary read server when using readpreference NEAREST

/Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/db.js:297
throw err;
^
TypeError: Cannot read property 'port' of undefined
at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/test/tests/repl_set/read_preferences_spec_tests.js:186:40
at /Volumes/DataHFS/Users/vaseker/Documents/node/ytools/node_modules/mongodb/lib/mongodb/db.js:294:11
at process._tickCallback (node.js:415:13)

I'll try to resolve this

Reply to this email directly or view it on GitHubhttps://github.com//pull/1159#issuecomment-40104972
.

christkv · 2014-04-10T20:46:18Z

tests should be fixed now in master

vaseker · 2014-04-11T13:19:14Z

Can you explain points to select server with 'undefined' ping? Is it bug or feature?
Test fails because of line 113 : I removed from 'finalCandidates' list servers with 'undef' ping.

I cannot understand default behavior like this (blocked 27019 port):

Ping strategy selection order for tags
localhost:27020 = 1 ms
localhost:27019 = undefined ms
picked server localhost:27019

christkv · 2014-04-11T14:13:01Z

yeah this patch won't work because there will be an interval between the actual connect and the first ping happening and thus all instances are 'undefined'. the code filters out all of the servers thus failing all operations until there is a ping.

christkv · 2014-04-11T14:38:56Z

this commit filters out any non-connected servers 85a92fc

vaseker · 2014-04-11T15:06:39Z

So, in _failTimer, instead of

__serverInstance.runtimeStats['pingMs'] = undefined;

I must mark server as 'non-connected'? I think it is overflow in 'ping strategy' scope.

The point is to quickly detect fallen replica server, prevent send new queries to it, and continue quering when connection restores.

vaseker · 2014-04-11T15:27:23Z

Maybe, better way is to trigger 'timeout' event faster?
Is it possible to configure terms to fire 'timeout' event?
I didn't find the source of 'timeout' event. Is it net.Socket timeout?

Then 85a92fc and connection/server/connectionPool.on("timeout", cb) solve 'quering unavalable server' problem.

Playing with server unavailability marks.

vaseker · 2014-04-14T10:39:46Z

I fixed code, and successfully test it, but when I retried test, it fails.
Cannot understand why.
Branch 1.4 cannot pass test:

STARTING 0 :: 6 :: Should correctly receive reconnect event after primary selection and buffering

/Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/connection/base.js:245
        throw message;      
              ^
AssertionError: 0 == 1
    at equal (/Users/vaseker/Documents/www/ytools/node_modules/mongodb/node_modules/integra/lib/test_control.js:37:14)
    at /Users/vaseker/Documents/www/ytools/node_modules/mongodb/test/tests/repl_set/reconnect_tests.js:329:16
    at /Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/collection/commands.js:52:5
    at /Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/db.js:1129:7
    at /Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/db.js:1843:9
    at Server.Base._callHandler (/Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/connection/base.js:445:41)
    at /Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/connection/server.js:468:18
    at MongoReply.parseBody (/Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
    at null.<anonymous> (/Users/vaseker/Documents/www/ytools/node_modules/mongodb/lib/mongodb/connection/server.js:426:20)
    at EventEmitter.emit (events.js:95:17)

vaseker · 2014-04-15T06:53:28Z

Moved to clean #1161
Tested with pingTimeout '100' and 'null'

Added support for unavailable replica servers

ba2956a

vaseker added 2 commits April 11, 2014 20:37

Fix for unavailable replica servers

86b1ec6

Merge branch '1.4' into pingStrategyLongs

c81e3cf

merged with disconnected servers

7f11c83

vaseker added 2 commits April 11, 2014 21:32

Temp.

1a7e36b

Playing with server unavailability marks.

Replica server availability check

93083fb

vaseker closed this Apr 14, 2014

vaseker deleted the pingStrategyLongs branch April 14, 2014 09:38

vaseker restored the pingStrategyLongs branch April 14, 2014 09:38

vaseker deleted the pingStrategyLongs branch April 14, 2014 09:42

vaseker restored the pingStrategyLongs branch April 14, 2014 10:38

vaseker reopened this Apr 14, 2014

vaseker closed this Apr 15, 2014

vaseker deleted the pingStrategyLongs branch April 15, 2014 06:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for unavailable replica servers #1159

Added support for unavailable replica servers #1159

vaseker commented Apr 10, 2014

christkv commented Apr 10, 2014

vaseker commented Apr 10, 2014

christkv commented Apr 10, 2014

vaseker commented Apr 10, 2014

vaseker commented Apr 10, 2014

christkv commented Apr 10, 2014

christkv commented Apr 10, 2014

vaseker commented Apr 11, 2014

christkv commented Apr 11, 2014

christkv commented Apr 11, 2014

vaseker commented Apr 11, 2014

vaseker commented Apr 11, 2014

vaseker commented Apr 14, 2014

vaseker commented Apr 15, 2014

Added support for unavailable replica servers #1159

Added support for unavailable replica servers #1159

Conversation

vaseker commented Apr 10, 2014

christkv commented Apr 10, 2014

vaseker commented Apr 10, 2014

christkv commented Apr 10, 2014

vaseker commented Apr 10, 2014

vaseker commented Apr 10, 2014

christkv commented Apr 10, 2014

christkv commented Apr 10, 2014

vaseker commented Apr 11, 2014

christkv commented Apr 11, 2014

christkv commented Apr 11, 2014

vaseker commented Apr 11, 2014

vaseker commented Apr 11, 2014

vaseker commented Apr 14, 2014

vaseker commented Apr 15, 2014