Multiple OutputURIs for MongoS [comSysto] HADOOP-59 #65

Closed
wants to merge 4 commits into
from

Conversation

Projects
None yet
2 participants
@devproof

Hi,

I added the capability to add multiple output URIs. Now its possible to support more than one MongoS for the output. Round robin is used for distribution. The API is backwards compatible.

How to use it:

MongoConfigUtil.setOutputURI(conf, "mongodb://mongos1/mydb.mycollection mongodb://mongos2/mydb.mycollection"); (String is splitted by StringUtils.split)

It is possible to use the old way to too:
MongoConfigUtil.setOutputURI(conf, "mongodb://mymongo/mydb.mycollection");

Unfortunately, we have to define the database and collection name twice due to the MongoURI Format.

Kind regards
Carsten Hufe

  • comSysto -
@mpobrien

This comment has been minimized.

Show comment Hide comment
@mpobrien

mpobrien Mar 15, 2013

Member

Hi Carsten,
It would not allow me to merge the pull request automatically but I've merged it manually to master, with some changes:

  • modify the build script to add apache commons lang as a dependency (needed for StringUtils.split on cdh3)
  • added a test case to try a job with 2 output uris.

Commit is here:
cb5596d

Thank you so much!

Member

mpobrien commented Mar 15, 2013

Hi Carsten,
It would not allow me to merge the pull request automatically but I've merged it manually to master, with some changes:

  • modify the build script to add apache commons lang as a dependency (needed for StringUtils.split on cdh3)
  • added a test case to try a job with 2 output uris.

Commit is here:
cb5596d

Thank you so much!

@mpobrien mpobrien closed this Mar 15, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment