Create default buckets via environment variables in docker

[christopherobin]

I am using minio mainly has a throwaway cache for GitLab CI runners, the container is living as a docker service in a swarm cluster and may at any point be restarted on another machine, since it is used only as a cache setting up distributed storage for persistence seems overkill.

The issue I'm running in is that on restart my bucket will be gone, preventing new CI jobs from creating cache entry, it would be nice to have a new environment variable like MINIO_DEFAULT_BUCKETS that would create the folders/buckets when the container start.

[vadmeste]

I am using minio mainly has a throwaway cache for GitLab CI runners,

I guess that you are running Minio in standalone FS mode.

it would be nice to have a new environment variable like MINIO_DEFAULT_BUCKETS that would create the folders/buckets when the container start.

In case of standalone FS mode, you can create some empty directories before running Minio:

$ mkdir /tmp/backend
$ mkdir /tmp/backend/bucket
$ minio server /tmp/backend

You might think this is not a clean way to do, but actually Minio FS is prepared to work the first time with existing data.

[krishnasrinivas]

Mapping a host volume for persistence is not an option for you @christopherobin ?

docker run -p 9000:9000 --name minio1 \
  -v /mnt/export/minio1:/export \
  -v /mnt/config/minio1:/root/.minio \
  minio/minio server /export

[harshavardhana]

@christopherobin alternative docker native way is to do this with docker-compose.yml

version: "2"
services:
  minio:
    image: minio/minio
    ports:
      - "9000:9000"
    volumes:
      - ./test/.minio/data:/export
      - ./test/.minio/config:/root/.minio
    environment:
      - "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE"
      - "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
    command: server /export

  createbuckets:
    image: minio/mc
    depends_on:
      - minio
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc config host add myminio http://minio:9000 AKIAIOSFODNN7EXAMPLE wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY;
      /usr/bin/mc rm -r --force myminio/somebucketname;
      /usr/bin/mc mb myminio/somebucketname;
      /usr/bin/mc policy download myminio/somebucketname;
      exit 0;
      "

[christopherobin]

@krishnasrinivas not really since we run in swarm and I don't want to lock the container to a specific host
@harshavardhana Yeah, in the end I overrided the command from the minio/minio container and used the solution from @vadmeste for now

[harshavardhana]

Feel free to close the issue if you have no further questions @christopherobin - Thanks.

[LoicMahieu]

Just in case someone is interested, here is a snippet for a "one line" docker only bucket creation:

export MINIO_DOCKER_NAME=test-minio
export MINIO_ACCESS_KEY=foobar
export MINIO_SECRET_KEY=barfoobarfoo
export MINIO_BUCKET=barfoobarfoo

docker run --name $MINIO_DOCKER_NAME -d -p 9000:9000 -e MINIO_ACCESS_KEY=$MINIO_ACCESS_KEY -e MINIO_SECRET_KEY=$MINIO_SECRET_KEY minio/minio server /data
docker run --rm --link $MINIO_DOCKER_NAME:minio -e MINIO_BUCKET=$MINIO_BUCKET --entrypoint sh minio/mc -c "\
  while ! nc -z minio 9000; do echo 'Wait minio to startup...' && sleep 0.1; done; \
  sleep 5 && \
  mc config host add myminio http://minio:9000 \$MINIO_ENV_MINIO_ACCESS_KEY \$MINIO_ENV_MINIO_SECRET_KEY && \
  mc rm -r --force myminio/\$MINIO_BUCKET || true && \
  mc mb myminio/\$MINIO_BUCKET && \
  mc policy download myminio/\$MINIO_BUCKET \
"

[christopherobin]

Also if it helps, here is the docker stack I ended up using (not the cleanest but works for what I'm doing with it):

version: "3.2"
services:
  minio:
    image: minio/minio:latest
    entrypoint: sh
    command: -c 'mkdir -p /export/gitlab && /usr/bin/minio server /export'
    environment:
          MINIO_ACCESS_KEY: "{{ minio_access_key }}"
          MINIO_SECRET_KEY: "{{ minio_secret_key }}"
    deploy:
      restart_policy:
        condition: on-failure

[hoto]

This issue has been closed. Does it mean this is not gonna end up on a roadmap?

[harshavardhana]

Yes of course - this is working as expected and also explained how it can be done using mc.

[maejimayuto]

#4769 (comment)

if you want to change bucket policy, maybe you have to this.
you forget set .

    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc config host add myminio http://minio:9000 AKIAIOSFODNN7EXAMPLE wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY;
      /usr/bin/mc rm -r --force myminio/somebucketname;
      /usr/bin/mc mb myminio/somebucketname;
      /usr/bin/mc policy set download myminio/somebucketname;
      exit 0;
      "

ref:
https://docs.min.io/docs/minio-client-complete-guide#policy