Error during Docker rolling update: Expected same MINIO_ environment variables and values

[moljor]

Hi,

We have a setup with a cluster of 4 VM's each running their own Minio Docker container.
I'm trying to update from version RELEASE.2021-08-25T00-41-18Z to anything above but keep receiving the following error on the first updated server1 (example name):

Error: https://server2:9000/data/disk1 as has incorrect configuration: Expected same MINIO_ environment variables and values (*fmt.wrapError)
1: cmd/server-main.go:533:cmd.serverMain()

I've compared the environment variables through docker inspect and there's no difference. I even updated the PATH variable as this changed during the update but the issue persists.

Following are the ENV variables in the containers (obviously changed the values a bit):

"container=oci",
"MINIO_ACCESS_KEY_FILE=access_key",
"MINIO_CONFIG_ENV_FILE=config.env"
"MINIO_KMS_KES_CERT_FILE=/location/cert",
"MINIO_KMS_KES_ENDPOINT=https://localhost:7373",
"MINIO_KMS_KES_KEY_FILE=/location/key",
"MINIO_KMS_KES_KEY_NAME=key-name",
"MINIO_KMS_SECRET_KEY_FILE=kms_master_key",
"MINIO_REGION_NAME=westeurope",
"MINIO_ROOT_PASSWORD_FILE=/location/file",
"MINIO_ROOT_USER_FILE=/location/file",
"MINIO_SECRET_KEY_FILE=secret_key",
"MINIO_UPDATE_MINISIGN_PUBKEY=SOMEKEY",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin",

Rolling back to RELEASE.2021-08-25T00-41-18Z works fine. And updating all of the containers with downtime also works but I'd like to be able to do a rolling update for obvious reasons in production.

Any ideas what might be going wrong here?

Thanks!

[harshavardhana]

Rolling back to RELEASE.2021-08-25T00-41-18Z works fine. And updating all of the containers with downtime also works but I'd like to be able to do a rolling update for obvious reasons in production.

Just upgrade all servers it takes 3-4secs there is no downtime as such, all clients retry their calls - S3 API is HTTP based.

[moljor]

Thanks for an answer!

Any idea though why that env check is being tripped?
Does the old version perhaps not return the env vars in the ServerSystemConfig object so it doesn't exist hence the mismatch?