Feature request: replica set support #12

Open
activeprospect opened this Issue Oct 25, 2010 · 18 comments

Comments

Projects
None yet
10 participants
@ghost

ghost commented Oct 25, 2010

It would be great to support connecting to replica sets.

http://www.mongodb.org/display/DOCS/Connecting+Drivers+to+Replica+Sets

@krobertson

This comment has been minimized.

Show comment
Hide comment
@krobertson

krobertson Dec 6, 2010

Any plans for this?

Any plans for this?

@tobsch

This comment has been minimized.

Show comment
Hide comment
@tobsch

tobsch Jun 24, 2011

interested too!

tobsch commented Jun 24, 2011

interested too!

@rcreasey

This comment has been minimized.

Show comment
Hide comment

+1

@alebsack

This comment has been minimized.

Show comment
Hide comment

+1

@cypriss

This comment has been minimized.

Show comment
Hide comment
@cypriss

cypriss Apr 5, 2012

I'd love tihs

cypriss commented Apr 5, 2012

I'd love tihs

@fl00r

This comment has been minimized.

Show comment
Hide comment
@fl00r

fl00r Jan 29, 2013

Just implemented this feature in my fork, going to check it out in production

fl00r commented Jan 29, 2013

Just implemented this feature in my fork, going to check it out in production

@ivobenedito

This comment has been minimized.

Show comment
Hide comment
@ivobenedito

ivobenedito Feb 5, 2013

Hi @fl00r,
Have you tested this out in a prod environment?
When do you think it will be ready for a PR?

Hi @fl00r,
Have you tested this out in a prod environment?
When do you think it will be ready for a PR?

@fl00r

This comment has been minimized.

Show comment
Hide comment
@fl00r

fl00r Feb 5, 2013

@ivobenedito, a week in prod. While mongodb master get down, client successfully reconnected

fl00r commented Feb 5, 2013

@ivobenedito, a week in prod. While mongodb master get down, client successfully reconnected

@ivobenedito

This comment has been minimized.

Show comment
Hide comment
@ivobenedito

ivobenedito Feb 5, 2013

@fl00r Cool ;)!! I'm mounting a ReplicaSet on my localhost to also make some failover tests.

@fl00r Cool ;)!! I'm mounting a ReplicaSet on my localhost to also make some failover tests.

@fl00r

This comment has been minimized.

Show comment
Hide comment
@fl00r

fl00r Feb 5, 2013

It will be good to get some feedback :)
Ehmm.. By the way, replica set connection should be established by new_replicas method

EM::Mongo::Connection.new_replicas([host:port, host:port, ...], timeout, reconnect_in: 1000)

fl00r commented Feb 5, 2013

It will be good to get some feedback :)
Ehmm.. By the way, replica set connection should be established by new_replicas method

EM::Mongo::Connection.new_replicas([host:port, host:port, ...], timeout, reconnect_in: 1000)
@ivobenedito

This comment has been minimized.

Show comment
Hide comment
@ivobenedito

ivobenedito Feb 7, 2013

@fl00r Thanks for showing the syntax usage. Did some tests with it yesterday.
Meanwhile, what do you think on having a more solid API, for example:

EM::Mongo::Connection.new # simple connector
EM::Mongo::ReplicaSetConnection.new # replset connector

# currently, the original mongo driver updated to this
Mongo::MongoClient.new # simple connector
Mongo::MongoReplicaSetClient.new # replset connector

@fl00r Thanks for showing the syntax usage. Did some tests with it yesterday.
Meanwhile, what do you think on having a more solid API, for example:

EM::Mongo::Connection.new # simple connector
EM::Mongo::ReplicaSetConnection.new # replset connector

# currently, the original mongo driver updated to this
Mongo::MongoClient.new # simple connector
Mongo::MongoReplicaSetClient.new # replset connector
@ivobenedito

This comment has been minimized.

Show comment
Hide comment
@ivobenedito

ivobenedito Feb 7, 2013

@fl00r Also, think I might have found a bug.

I'm using a ReplicaSet with 3 members, everytime I shutdown the primary and a secondary from the remaining 2 gets elected to master, the client works fine.

Meanwhile, if I shutdown another member (primary) and only leave 1 active (secondary), mongo doesn't elect the secondary to primary without an arbiter, which means the ReplSet stops working.
This is how mongo works, and the app isn't able to fetch anything from Mongo because there is no primary node. (makes sense)

Then, If I bring another member in again, a primary is elected but the app/driver can't find it anymore.

Have you covered/tested this use case?

@fl00r Also, think I might have found a bug.

I'm using a ReplicaSet with 3 members, everytime I shutdown the primary and a secondary from the remaining 2 gets elected to master, the client works fine.

Meanwhile, if I shutdown another member (primary) and only leave 1 active (secondary), mongo doesn't elect the secondary to primary without an arbiter, which means the ReplSet stops working.
This is how mongo works, and the app isn't able to fetch anything from Mongo because there is no primary node. (makes sense)

Then, If I bring another member in again, a primary is elected but the app/driver can't find it anymore.

Have you covered/tested this use case?

@fl00r

This comment has been minimized.

Show comment
Hide comment
@fl00r

fl00r Feb 7, 2013

Right now driver tries to reconnect few times with timeout delay and then calls on_unbind callback (which is do nothing by default).

In production you should raise an error and stop your application in case if after some retries client can't reconnect. Then you will get some alerts from production and then somehow solve your problem and start your application again (we use runitd for supervising our applications).

So you should pass on_unbind: proc{ fail "Can't reconnect blah blah blah" } option. Or I can add support of infinite reconnects, which looks like a bad option.

About naming. In my first iteration I've created ReplicaSetConnection class for connection :) so, now I think it is good idea to return back

fl00r commented Feb 7, 2013

Right now driver tries to reconnect few times with timeout delay and then calls on_unbind callback (which is do nothing by default).

In production you should raise an error and stop your application in case if after some retries client can't reconnect. Then you will get some alerts from production and then somehow solve your problem and start your application again (we use runitd for supervising our applications).

So you should pass on_unbind: proc{ fail "Can't reconnect blah blah blah" } option. Or I can add support of infinite reconnects, which looks like a bad option.

About naming. In my first iteration I've created ReplicaSetConnection class for connection :) so, now I think it is good idea to return back

@fl00r

This comment has been minimized.

Show comment
Hide comment
@fl00r

fl00r Mar 28, 2013

Hi. As far as em-mongo isn't supported enough I've begun to write new MongoDB Client on EventMachine.
Protocol and some common stuf is already implemented. ReplicaSets support will be added till Monday.

https://github.com/fl00r/monga

If somebody interested - welcome!

fl00r commented Mar 28, 2013

Hi. As far as em-mongo isn't supported enough I've begun to write new MongoDB Client on EventMachine.
Protocol and some common stuf is already implemented. ReplicaSets support will be added till Monday.

https://github.com/fl00r/monga

If somebody interested - welcome!

@bcg

This comment has been minimized.

Show comment
Hide comment
@bcg

bcg Apr 5, 2013

Owner

@fl00r if you have any interest in taking over em-mongo, or just taking the name once you have feature parity, I would be more than happy to hand over the keys (to the gem as well). At the very least let me know when you think monga its stable for production use and I will make mention of it in the README.

I haven't used MongoDB in a long, long time and probably never will.

cc: @ivobenedito

Owner

bcg commented Apr 5, 2013

@fl00r if you have any interest in taking over em-mongo, or just taking the name once you have feature parity, I would be more than happy to hand over the keys (to the gem as well). At the very least let me know when you think monga its stable for production use and I will make mention of it in the README.

I haven't used MongoDB in a long, long time and probably never will.

cc: @ivobenedito

@fl00r

This comment has been minimized.

Show comment
Hide comment
@fl00r

fl00r Apr 8, 2013

Hello! Somehow github removed all my notifications :/
It is a good idea to not produce many clients and to maintain only one.

fl00r commented Apr 8, 2013

Hello! Somehow github removed all my notifications :/
It is a good idea to not produce many clients and to maintain only one.

@fl00r

This comment has been minimized.

Show comment
Hide comment
@fl00r

fl00r Apr 18, 2013

I've rewritten my client to support all kind of interfaces: blocking (over TCPSocket), synchronous (over Fibers on EventMachine) and asynchronous (over EventMachine). But API has changed because I've used callbacks instead of Deferrable. Going to rewrite Wiki and Readme soon

fl00r commented Apr 18, 2013

I've rewritten my client to support all kind of interfaces: blocking (over TCPSocket), synchronous (over Fibers on EventMachine) and asynchronous (over EventMachine). But API has changed because I've used callbacks instead of Deferrable. Going to rewrite Wiki and Readme soon

@dCSeven

This comment has been minimized.

Show comment
Hide comment
@dCSeven

dCSeven Mar 13, 2018

Collaborator

@fl00r I've seen that you've implemented replica set support on your repo (some time ago, I know).

If you like to, it would be a great thing to merge your commits into this here.

Collaborator

dCSeven commented Mar 13, 2018

@fl00r I've seen that you've implemented replica set support on your repo (some time ago, I know).

If you like to, it would be a great thing to merge your commits into this here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment