-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Arango cluster in docker #21
Comments
@virus2016 yes, you can build a cluster with different numbers of data nodes and coordinator nodes. We provide different scripts to deploy ArangoDB in the cloud. For details see https://github.com/ArangoDB/deployment There are scripts to deploy ArangoDB on DigitalOcean, AWS, Azure and Google GCE. If you want to test a cluster on your local machine, you can follow the instruction given in https://github.com/m0ppers/mesos-cluster This will setup a cluster locally using mini-mesos and docker. Is that, what you had in mind? |
Thanks Frank! Is there a way of self discovery like elasticsearch? And do the node only replicate data i.e. each node has the same data on it? Many thanks, -------- Original message -------- @virus2016https://github.com/virus2016 yes, you can build a cluster with different numbers of data nodes and coordinator nodes. We provide different scripts to deploy ArangoDB in the cloud. For details see https://github.com/ArangoDB/deployment There are scripts to deploy ArangoDB on DigitalOcean, AWS, Azure and Google GCE. If you want to test a cluster on your local machine, you can follow the instruction given in https://github.com/m0ppers/mesos-cluster This will setup a cluster locally using mini-mesos and docker. Is that, what you had in mind? You are receiving this because you were mentioned. |
The replication of data depends on your configuration. Normally you would set up a cluster, in which the collections are shared. I. e. each node only contains part of the data. In the standard setup, every shared is stored on two servers for resilience. You could also configure the cluster in such a way, that there is only one shard in which case each server will have the same data on it. What is your use-case? Do you scale because you have so much data, that it does not fit on a single server? Or do you need to scale because you need more CPU power for queries? Currently, we are using Mesos for discovery. Mesos handles the resource management for us and sets up service locators to automatically find the service endpoints. We are currently working hard to simplify the process even further. In the near future, we will also support Kubernetes. In which environment (cloud or otherwise) are you planning to us ArangoDB? Best |
Hey Frank, Thanks for spending time messaging me back. We are currently using Rancher UI which is a docker orchestration tool (Awesome BTW). I am looking to scale for the CPU side. The data isn’t that big. So let’s make it simple. I am wanting to create 10 “data” and 3 “client” containers. How would I go about doing that purely from the command line i.e. docker run…. Thanks again for your help on this. |
I'm not familiar with Rancher UI - to be honest. How does fail-over and restart of failed containers? Is there anything elaborated like Apache Mesos Resource scheduling? Or does it "just" start a number of containers? If so, how are IP addresses of these containers published? In General: you can start ArangoDB as a bunch of docker containers manually. A good starting point is https://github.com/arangodb/deployment/blob/master/Docker/ArangoDBClusterWithDocker.sh this script will deploy ArangoDB as docker containers. Best |
Hey Frank, Now that 3.0 is out, is there away to create a cluster from the docker image? Cheers, |
Hi @virus2016, https://github.com/arangodb/arangodb/blob/devel/README_maintainers.md#arangodb-on-mesos (please note that this is a temporary instance, which will vanish if you terminate the controlling process, but it should serve well how to get started) |
Hi @dothebart, I've used the mesos setup but I'm looking to try and simplify this deployment. Our customers will not run Mesos... I'm I right in thinking that each role should be a separate container in docker? DBServer, agency and coordinator? If so, can't we just do docker run x3 and what commands would they be? Cheers, |
Hi @virus2016, meanwhile we used the time to finish the documentation for the cluster setup: |
Hey can't this be deployed with docker swarm? |
Another quick question, could someone create a docker-compose file for deployment of a cluster? That would help! Cheers, |
@fceller Can you help? Cheers, |
we currently have https://github.com/arangodb/arangodb-mesos-docker - maybe that would be a good starting point for you? Since we don't exactly know whats expected there you would need to help us out here. If you want more interactivity to solve this, maybe join our slack channel so we can discuss this further how you could solve this? |
Hi @virus2016 if you stay with us a little time longer, I think we can just do these things very easily. We are currently streamlining the boot process considerably so that starting in a non-mesos environment becomes almost trivial. |
Thanks guys, this will really help us! In the meantime, here is my docker-compose. Can you have a look over it and see if I am doing the compose right? `version: "2" services:
|
I am starting to look now. Initial observation: I think your problems might be related to Docker/port issues. I do not know docker-compose too well, but as far as I see, your agency entry does not make sure that the agency is visible on port 5001 outside its docker container. I will experiment to find out. |
The agency should work provided you forward port 5001 to outside the docker container, either by adding |
I think I withdraw all my comments. Here is the file
|
I think this setup is all right. It is good that only the coordinator ports are exposed on the host's main interface. I like that the others find themselves by using docker-composes' name lookup. Obviously, as docker-compose, it only works on a single machine, but for testing this is sensible. I will suggest to put this as an additional section in the Deployment chapter of the manual, because I think this can be useful for other people. |
@neunhoef I am having some issues with the cluster in docker. Maybe you can point me in the right direction. I am running the following:
It all starts up and works fine. I add a DB and it sometimes crashes due to a replication err. Now if I am lucky to create the DB, I then create a collection. 3 shards 2 replications. I then add some data (250,000 docs) then I get this err: ←[36;1mdb1_1 |←[0m 2016-12-08T00:47:35Z [1] INFO {replication} connected to master at tcp://db2:8529, id 122177945464438, version 3.1, last log tick 2102976 I am running docker for windows. Containers: 8 Any help would be appreciated. Thanks, |
Ill wait until Frank comes back to me. Thanks guys! |
The now recommended way to launch clusters is using the ArangoDBStarter - please see https://github.com/arangodb-helper/arangodb - it also supports using this docker container. |
@fceller You mentioned kubernetes support in you comment. I know it has been a while so I was wondering If there is some official support for kubernetes right now. |
Hi @haseebnaseem - please have a look at this screencast by our community member @sbaugher https://www.arangodb.com/tech-talks/ |
If anyone is looking for a way to run ArangoDB in Swarm here's an example configuration that allows you to scale Agents, DBServers and Controllers independently with |
Thanks for sharing @JonDum, will add a link to your repo to our docs: arangodb/docs#542 |
Hi guys,
Bit of help please. I'm wanting to build an arrango cluster in docker. 10 data nodes and 3 coordinators. Firstly, is this possible? Or I'm I doing something wrong... (Must be the elasticsearch within me)
Thanks in advance
The text was updated successfully, but these errors were encountered: