New server startup failure - using root disk as path for data

[sbwise-airbnb]

NOTE

If this case is urgent, please subscribe to Subnet so that our 24/7 support team may help you faster.

The newest version of minio released on 2022-06-02 (https://dl.min.io/server/minio/release/linux-amd64/minio) causes a startup error when you provide a data directory that is on the root disk. This error did not exist for the version released on 2022-05-26 (https://dl.min.io/server/minio/release/linux-amd64/archive/minio.RELEASE.2022-05-26T05-48-41Z)

Expected Behavior

Server should start up successfully

root@98c67a0d2998:/# ./minio server /opt --address ":9000" --console-address ":9001"
API: http://172.17.0.2:9000  http://127.0.0.1:9000
RootUser: minioadmin
RootPass: minioadmin
Finished loading IAM sub-system (took 0.0s of 0.0s to load data).

Console: http://172.17.0.2:9001 http://127.0.0.1:9001
RootUser: minioadmin
RootPass: minioadmin

Command-line: https://docs.min.io/docs/minio-client-quickstart-guide
   $ mc alias set myminio http://172.17.0.2:9000 minioadmin minioadmin

Documentation: https://docs.min.io

Current Behavior

Server fails to start up

root@857d6d701208:/# ./minio server /opt --address ":9000" --console-address ":9001"
Formatting 1st pool, 1 set(s), 1 drives per set.

API: SYSTEM()
Time: 17:40:01 UTC 06/02/2022
Error: Disk `/opt` is part of root disk, will not be used (*errors.errorString)
       8: internal/logger/logger.go:278:logger.LogIf()
       7: cmd/erasure-sets.go:1243:cmd.markRootDisksAsDown()
       6: cmd/format-erasure.go:798:cmd.initFormatErasure()
       5: cmd/prepare-storage.go:198:cmd.connectLoadInitFormats()
       4: cmd/prepare-storage.go:282:cmd.waitForFormatErasure()
       3: cmd/erasure-server-pool.go:66:cmd.newErasureServerPools()
       2: cmd/server-main.go:668:cmd.newObjectLayer()
       1: cmd/server-main.go:518:cmd.serverMain()
Formatting 1st pool, 1 set(s), 1 drives per set.

API: SYSTEM()
Time: 17:40:01 UTC 06/02/2022
Error: Disk `/opt` is part of root disk, will not be used (*errors.errorString)
       8: internal/logger/logger.go:278:logger.LogIf()
       7: cmd/erasure-sets.go:1243:cmd.markRootDisksAsDown()
       6: cmd/format-erasure.go:798:cmd.initFormatErasure()
       5: cmd/prepare-storage.go:198:cmd.connectLoadInitFormats()
       4: cmd/prepare-storage.go:302:cmd.waitForFormatErasure()
       3: cmd/erasure-server-pool.go:66:cmd.newErasureServerPools()
       2: cmd/server-main.go:668:cmd.newObjectLayer()
       1: cmd/server-main.go:518:cmd.serverMain()
ERROR Unable to initialize backend: disk not found
root@857d6d701208:/#

Possible Solution

Use the version released on 2022-05-26 instead: https://dl.min.io/server/minio/release/linux-amd64/archive/minio.RELEASE.2022-05-26T05-48-41Z

Steps to Reproduce (for bugs)

1. Start a docker container using the image ubuntu:18.04
2. Download minio from https://dl.min.io/server/minio/release/linux-amd64/minio to the running docker container
3. Start mino using the command ./minio server /opt --address ":9000" --console-address ":9001"

Context

CI job is now failing due to minio server failing to start

Regression

Yes this is a regression

Release https://github.com/minio/minio/tree/RELEASE.2022-06-02T02-11-04Z

Your Environment

• Version used (minio --version): minio version RELEASE.2022-06-02T02-11-04Z
• Server setup and configuration: Docker container from https://hub.docker.com/layers/ubuntu/library/ubuntu/18.04/images/sha256-ec5715e4fd292b212edd9062c1abe247298ed3acf87b9555b55cc9b16e7f628a?context=explore
• Operating System and version (uname -a): Linux ff18785e2a18 5.10.104-linuxkit #1 SMP Thu Mar 17 17:08:06 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

[harshavardhana]

CI=true or MINIO_CI_CD=true - if you are using root disk make sure to let MinIO know that this is a CI/CD environment.

[sbwise-airbnb]

I see now there are multiple issues reported for this: https://github.com/minio/minio/search?q=MINIO_CI_CD&type=issues

Maybe a blurb in the README might help?

[harshavardhana]

I see now there are multiple issues reported for this: https://github.com/minio/minio/search?q=MINIO_CI_CD&type=issues

Maybe a blurb in the README might help?

Root Disks we do not expect that you would use, let me add a line in README.md cc // @ravindk89

[majetideepak]

I see that the file .minio.sys/format.json is missing when using the latest release (RELEASE.2022-06-02T16-16-26Z) on a new data directory. This is causing the error posted here as well as existing data not being detected. Is this expected?

[harshavardhana]

I see that the file .minio.sys/format.json is missing when using the latest release (RELEASE.2022-06-02T16-16-26Z) on a new data directory. This is causing the error posted here as well as existing data not being detected. Is this expected?

Yes please read #14331 - existing data support is gone, and no new fresh FS mode is possible anymore. Older FS mode continues to work as-is if it finds older content with format.json that says its "FS" mode, newer deployments will move to single drive XL format - no pre-existing data is supported.

[majetideepak]

@harshavardhana thank you for your response.

[harshavardhana]

This is also fixed by #15037