Documentation on multihost deployment (Docker swarm mode)

[mporracindie]

It would be great to have a little doc on how to deploy a 2 or 3 nodes cluster and maybe even how to make in bigger on hot (while its running) without downtime.

Ive been researching a little on this and making some testing but cant get it to work with Docker Swarm. If i do i ll add a comment with the docker-compose.yml as a possible example.

Thanks!

[mporracindie]

After some try and error y managed to make it works with some caveats i still need to figure it out.

• Had to run it on docker 16.04 since more recent version dont support ulimits on docker-compose.yml files and run this:

echo -e "[Service]\nLimitMEMLOCK=infinity" | SYSTEMD_EDITOR=tee systemctl edit docker.service
systemctl daemon-reload
systemctl restart docker

(cant find the source, i ll link it later)

• I have to test how it scales without split brain situations and if it causes downtime.

Here are some usefull links and the docker-compose.yml:

https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/
https://docs.docker.com/engine/swarm/swarm-tutorial/deploy-service/
https://docs.docker.com/engine/reference/commandline/service_scale/

version: '3'
services:
  elastic-node1:
    image: amazon/opendistro-for-elasticsearch:0.7.0
    environment:
      - cluster.name=elastic-cluster
      - node.name=elastic-node1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms16g -Xmx16g"
      - opendistro_security.ssl.http.enabled=false
      - discovery.zen.minimum_master_nodes=2
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elastic-data1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
      - 9600:9600
    networks:
      - elastic-net
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
            - node.role == manager
  elastic-node2:
    image: amazon/opendistro-for-elasticsearch:0.7.0
    environment:
      - cluster.name=elastic-cluster
      - node.name=elastic-node2
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms16g -Xmx16g"
      - opendistro_security.ssl.http.enabled=false
      - "discovery.zen.ping.unicast.hosts=elastic-node1"
      - discovery.zen.minimum_master_nodes=2
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elastic-data2:/usr/share/elasticsearch/data
    networks:
      - elastic-net
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
            - node.role != manager

  kibana:
    image: amazon/opendistro-for-elasticsearch-kibana:0.7.0
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      ELASTICSEARCH_URL: http://elastic-node1:9200
    networks:
      - elastic-net
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
            - node.role == manager

volumes:
  elastic-data1:
  elastic-data2:

networks:
  elastic-net:

After the initial start i managed to grow it with:

docker service scale stackdemo_elastic-node2=2

And thats how far i got with the testing for now.
Hope it helps!

[oddlittlebird]

Hi there, technical writer here. I am new to open source and Elastic, but interested in helping with documentation.

[aetter]

Hi @oddlittlebird, I went ahead and created some "good first issue" issues here: https://github.com/opendistro/for-elasticsearch-docs/issues

If those are too daunting, another good way to get started is to pick a page and test it: does the sample code work, are there typos or other inaccuracies, is it missing useful content, etc.

[oddlittlebird]

Awesome, thank you!