Can't deploy on Docker Swarm #19610

asijoumi · 2024-04-25T06:07:06Z

NOTE

If this case is urgent, please subscribe to Subnet so that our 24/7 support team may help you faster.

Hi ! I used the docker-compose.yaml located on the documentation (https://github.com/minio/minio/blob/master/docs/orchestration/docker-compose/docker-compose.yaml) and change it a little bit. But I was unable to used it on docker swarm.

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

docker-compose.yml

version: '3.7'

x-minio-common: &minio-common
  image: quay.io/minio/minio:RELEASE.2024-04-18T19-09-19Z
  command: server --console-address ":9001" http://minio{1...4}/data
  healthcheck:
    test: ["CMD", "mc", "ready", "local"]
    interval: 5s
    timeout: 5s
    retries: 5

services:
  minio1:
    <<: *minio-common
    volumes:
      - minio1:/data
    networks:
      minio:
        aliases:
          - minio1
  minio2:
    <<: *minio-common
    volumes:
      - minio2:/data
    networks:
      minio:
        aliases:
          - minio2
  minio3:
    <<: *minio-common
    volumes:
      - minio3:/data
    networks:
      minio:
        aliases:
          - minio3
  minio4:
    <<: *minio-common
    volumes:
      - minio4:/data
    networks:
      minio:
        aliases:
          - minio4

volumes:
  minio1:
  minio2:
  minio3:
  minio4:

networks:
  minio:
    driver: overlay

To easily test it, you can use https://labs.play-with-docker.com

docker stack deploy -c minio.yaml minio

Context

Regression

Your Environment

Version used (minio --version):
Server setup and configuration:
Operating System and version (uname -a):

The text was updated successfully, but these errors were encountered:

jiuker · 2024-04-25T06:52:32Z

What's the error log ? @asijoumi

asijoumi · 2024-04-25T07:38:00Z

Hi @jiuker, I have this :

minio_minio1.1.r19hdu824jv3@worker2    | Error: Unable to resolve DNS for http://minio3/data: lookup minio3 on 127.0.0.11:53: no such host (*fmt.wrapError)
minio_minio1.1.r19hdu824jv3@worker2    |        host="minio3", elapsedTime="1 second elapsed"
minio_minio1.1.r19hdu824jv3@worker2    |       10: internal/logger/logonce.go:118:logger.(*logOnceType).logOnceIf()
minio_minio1.1.r19hdu824jv3@worker2    |        9: internal/logger/logonce.go:149:logger.LogOnceIf()
minio_minio1.1.r19hdu824jv3@worker2    |        8: cmd/logging.go:82:cmd.bootLogOnceIf()
minio_minio1.1.r19hdu824jv3@worker2    |        7: cmd/endpoint.go:869:cmd.PoolEndpointList.UpdateIsLocal()
minio_minio1.1.r19hdu824jv3@worker2    |        6: cmd/endpoint.go:1007:cmd.CreatePoolEndpoints()
minio_minio1.1.r19hdu824jv3@worker2    |        5: cmd/endpoint-ellipses.go:508:cmd.createServerEndpoints()
minio_minio1.1.r19hdu824jv3@worker2    |        4: cmd/server-main.go:344:cmd.serverHandleCmdArgs()
minio_minio1.1.r19hdu824jv3@worker2    |        3: cmd/server-main.go:687:cmd.serverMain.func2()
minio_minio1.1.r19hdu824jv3@worker2    |        2: cmd/server-main.go:512:cmd.bootstrapTrace()
minio_minio1.1.r19hdu824jv3@worker2    |        1: cmd/server-main.go:683:cmd.serverMain()

jiuker · 2024-04-25T11:41:07Z

I think that's network issue. Not for minio.

asijoumi · 2024-04-25T11:58:08Z

It seems strange that the exemple of the documentation won't work with Docker Swarm...

Ok, do you have an exemple of deployment of minio with docker-compose ?

jiuker · 2024-04-25T12:18:02Z

It seems strange that the exemple of the documentation won't work with Docker Swarm...

Ok, do you have an exemple of deployment of minio with docker-compose ?

You mentioned one could be ok.

asijoumi · 2024-04-25T13:42:08Z

@jiuker, it is not woking ...

I copied the docker-compose.yml from the documentation :

version: '3.7'

# Settings and configurations that are common for all containers
x-minio-common: &minio-common
  image: quay.io/minio/minio:RELEASE.2024-04-18T19-09-19Z
  command: server --console-address ":9001" http://minio{1...4}/data{1...2}
  expose:
    - "9000"
    - "9001"
  # environment:
    # MINIO_ROOT_USER: minioadmin
    # MINIO_ROOT_PASSWORD: minioadmin
  healthcheck:
    test: ["CMD", "mc", "ready", "local"]
    interval: 5s
    timeout: 5s
    retries: 5

# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
  minio1:
    <<: *minio-common
    hostname: minio1
    volumes:
      - data1-1:/data1
      - data1-2:/data2

  minio2:
    <<: *minio-common
    hostname: minio2
    volumes:
      - data2-1:/data1
      - data2-2:/data2

  minio3:
    <<: *minio-common
    hostname: minio3
    volumes:
      - data3-1:/data1
      - data3-2:/data2

  minio4:
    <<: *minio-common
    hostname: minio4
    volumes:
      - data4-1:/data1
      - data4-2:/data2

  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
      - "9001:9001"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
  data1-1:
  data1-2:
  data2-1:
  data2-2:
  data3-1:
  data3-2:
  data4-1:
  data4-2:

Docker compose and Docker Swarm are not the same. So you do have an example for Docker Swarm ?

jiuker · 2024-04-25T13:50:57Z

@jiuker, it is not woking ...

I copied the docker-compose.yml from the documentation :

version: '3.7'

# Settings and configurations that are common for all containers
x-minio-common: &minio-common
  image: quay.io/minio/minio:RELEASE.2024-04-18T19-09-19Z
  command: server --console-address ":9001" http://minio{1...4}/data{1...2}
  expose:
    - "9000"
    - "9001"
  # environment:
    # MINIO_ROOT_USER: minioadmin
    # MINIO_ROOT_PASSWORD: minioadmin
  healthcheck:
    test: ["CMD", "mc", "ready", "local"]
    interval: 5s
    timeout: 5s
    retries: 5

# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
  minio1:
    <<: *minio-common
    hostname: minio1
    volumes:
      - data1-1:/data1
      - data1-2:/data2

  minio2:
    <<: *minio-common
    hostname: minio2
    volumes:
      - data2-1:/data1
      - data2-2:/data2

  minio3:
    <<: *minio-common
    hostname: minio3
    volumes:
      - data3-1:/data1
      - data3-2:/data2

  minio4:
    <<: *minio-common
    hostname: minio4
    volumes:
      - data4-1:/data1
      - data4-2:/data2

  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
      - "9001:9001"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
  data1-1:
  data1-2:
  data2-1:
  data2-2:
  data3-1:
  data3-2:
  data4-1:
  data4-2:

Docker compose and Docker Swarm are not the same. So you do have an example for Docker Swarm ?

You need to check your host. Like firework or others.

klauspost · 2024-04-25T14:02:43Z

It seems like something is wrong with your environment, since minio1 hosts cannot be resolved.

asijoumi · 2024-04-25T14:23:30Z

It's strange ... You can test this example :

version: '3.7'

x-minio-common: &minio-common
  image: quay.io/minio/minio:RELEASE.2024-04-18T19-09-19Z
  command: server --console-address ":9001" http://minio_minio{1...4}/data
  networks:
    - minio
  healthcheck:
    test: ["CMD", "mc", "ready", "local"]
    interval: 5s
    timeout: 5s
    retries: 5

services:
  minio1:
    <<: *minio-common
    volumes:
      - minio1:/data
  minio2:
    <<: *minio-common
    volumes:
      - minio2:/data
  minio3:
    <<: *minio-common
    volumes:
      - minio3:/data
  minio4:
    <<: *minio-common
    volumes:
      - minio4:/data

  nginx:
    image: nginx:1.19.2-alpine
    networks:
      - minio
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

volumes:
  minio1:
  minio2:
  minio3:
  minio4:

networks:
  minio:
    driver: overlay
    attachable: true

After that run :

docker run --rm -it --network=minio_minio node:18-alpine sh

Then :

apk add curl

And :

curl http://minio_nginx

We got an answer from nginx .. It's the same docker stack. Docker was able to resolve the nginx domain name but not the minio ?

harshavardhana · 2024-04-25T15:53:07Z

The DNS issues are common in docker swarm. Use k8s with a kind cluster much easier and more robust.

Docker is good at containers not at orchestration.

asijoumi added community triage labels Apr 25, 2024

harshavardhana closed this as completed Apr 25, 2024

harshavardhana added working as intended and removed triage labels Apr 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't deploy on Docker Swarm #19610

Can't deploy on Docker Swarm #19610

asijoumi commented Apr 25, 2024 •

edited

jiuker commented Apr 25, 2024

asijoumi commented Apr 25, 2024

jiuker commented Apr 25, 2024

asijoumi commented Apr 25, 2024

jiuker commented Apr 25, 2024 •

edited

asijoumi commented Apr 25, 2024

jiuker commented Apr 25, 2024

klauspost commented Apr 25, 2024

asijoumi commented Apr 25, 2024 •

edited

harshavardhana commented Apr 25, 2024

Can't deploy on Docker Swarm #19610

Can't deploy on Docker Swarm #19610

Comments

asijoumi commented Apr 25, 2024 • edited

NOTE

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Context

Regression

Your Environment

jiuker commented Apr 25, 2024

asijoumi commented Apr 25, 2024

jiuker commented Apr 25, 2024

asijoumi commented Apr 25, 2024

jiuker commented Apr 25, 2024 • edited

asijoumi commented Apr 25, 2024

jiuker commented Apr 25, 2024

klauspost commented Apr 25, 2024

asijoumi commented Apr 25, 2024 • edited

harshavardhana commented Apr 25, 2024

asijoumi commented Apr 25, 2024 •

edited

jiuker commented Apr 25, 2024 •

edited

asijoumi commented Apr 25, 2024 •

edited