Enable config for NAS gateway mode

[harshavardhana]

Description

Enable config for NAS gateway mode

Motivation and Context

Starting with #7751 we don't store config
in etcd anymore, allow NAS to honor config
on disk.

How to test this PR?

Just use minio gateway nas ~/test and use mc admin config set commands.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• Fixes a regression (If yes, please add commit-id or PR # here)
• Documentation needed
• Unit tests needed
• Functional tests needed (If yes, add mint PR # here: )

[harshavardhana]

Ping @vadmeste @poornas

[codecov-io]

Codecov Report

Merging #7948 into master will increase coverage by 0.42%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #7948      +/-   ##
=========================================
+ Coverage   44.78%   45.2%   +0.42%     
=========================================
  Files         302     314      +12     
  Lines       38947   50416   +11469     
=========================================
+ Hits        17442   22791    +5349     
- Misses      19399   25515    +6116     
- Partials     2106    2110       +4

Impacted Files	Coverage Δ
cmd/gateway-main.go	13.97% <0%> (-7.29%)	⬇️
cmd/config.go	28.72% <0%> (-6.28%)	⬇️
cmd/disk-usage.go	78.94% <0%> (-21.06%)	⬇️
cmd/service.go	27.27% <0%> (-10.23%)	⬇️
pkg/madmin/utils.go	16.66% <0%> (-7.15%)	⬇️
cmd/auth-handler.go	62.19% <0%> (-7.1%)	⬇️
cmd/config-dir.go	40.74% <0%> (-7.09%)	⬇️
cmd/globals.go	55.88% <0%> (-7.09%)	⬇️
cmd/server-rlimit.go	54.16% <0%> (-5.84%)	⬇️
... and 277 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55d4eee...7b4ef13. Read the comment docs.

[poornas]

if user edits the backend config json instead of using mc admin config get/set [with the incoming PR) and leaves the config in an inconsistent state, it will neither get reloaded nor reported. Suggest logging if error in validating config.

[harshavardhana]

if user edits the backend config json instead of using mc admin config get/set [with the incoming PR) and leaves the config in an inconsistent state, it will neither get reloaded nor reported. Suggest logging if error in validating config.

If user edits backend he is on his own @poornas - config.json is considered like metadata maintained by MinIO we don't need to be paranoid about it.

[kannappanr]

LGTM

[minio-ops]

Mint Automation

Test	Result
mint-compression-xl.sh	✔️
mint-xl.sh	✔️
mint-compression-dist-xl.sh	✔️
mint-compression-fs.sh	✔️
mint-worm.sh	✔️
mint-fs.sh	✔️
mint-gateway-nas.sh	✔️
mint-large-bucket.sh	❌ more...
mint-dist-xl.sh	❌ more...

7948-54e0624/mint-large-bucket.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.59:31416
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp ae2a84029e79:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... done in 2 seconds
(2/14) Running aws-sdk-java tests ... done in 2 seconds
(3/14) Running aws-sdk-php tests ... done in 52 seconds
(4/14) Running aws-sdk-ruby tests ... done in 11 seconds
(5/14) Running awscli tests ... done in 2 minutes and 30 seconds
(6/14) Running healthcheck tests ... done in 1 seconds
(7/14) Running mc tests ... FAILED in 1 minutes and 49 seconds
{
  "name": "mc",
  "duration": "106109",
  "function": "test_put_object_with_storage_class",
  "status": "FAIL",
  "error": "/mint/run/core/mc/mc --config-dir /tmp/.mc-334 --quiet --no-color cp --storage-class REDUCED_REDUNDANCY /mint/data/datafile-1-MB myminio/mc-test-bucket-19282/mc-test-object-30468  >>> \n`/mint/data/datafile-1-MB` -> `myminio/mc-test-bucket-19282/mc-test-object-30468`\nmc: <ERROR> Failed to copy `/mint/data/datafile-1-MB`. Please reduce your request\nmc: <ERROR> Session safely terminated. To resume session `mc session resume bSTYLmmr`"
}

Executed 6 out of 14 tests successfully.

7948-54e0624/mint-dist-xl.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.56:32392
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp d0701361125f:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... done in 0 seconds
(2/14) Running aws-sdk-java tests ... done in 2 seconds
(3/14) Running aws-sdk-php tests ... done in 42 seconds
(4/14) Running aws-sdk-ruby tests ... done in 3 seconds
(5/14) Running awscli tests ... done in 1 minutes and 6 seconds
(6/14) Running healthcheck tests ... done in 0 seconds
(7/14) Running mc tests ... done in 40 seconds
(8/14) Running minio-dotnet tests ... done in 37 seconds
(9/14) Running minio-go tests ... done in 1 minutes and 12 seconds
(10/14) Running minio-java tests ... FAILED in 45 seconds
{
  "name": "minio-java",
  "function": "listIncompleteUploads(String bucketName, String prefix)",
  "duration": 985,
  "status": "FAIL",
  "error": "error occurred\nErrorResponse(code=SlowDown, message=Please reduce your request, bucketName=minio-java-test-1dsoeoi, objectName=minio-java-test-1tlltg1, resource=/minio-java-test-1dsoeoi/minio-java-test-1tlltg1, requestId=15B4CB6A0BF922B1, hostId=a3b45ab5-376b-4327-b853-2df54aadff77)\nrequest={method=DELETE, url=http://72.28.97.56:32392/minio-java-test-1dsoeoi/minio-java-test-1tlltg1?uploadId=bcf891f9-8138-441c-97a7-000db254a859, headers=Host: 72.28.97.56:32392\nUser-Agent: MinIO (amd64; amd64) minio-java/dev\nx-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date: 20190725T235026Z\nAuthorization: AWS4-HMAC-SHA256 Credential=*REDACTED*/20190725/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=*REDACTED*\n}\nresponse={code=503, headers=Accept-Ranges: bytes\nContent-Length: 363\nContent-Security-Policy: block-all-mixed-content\nContent-Type: application/xml\nRetry-After: 120\nServer: MinIO/DEVELOPMENT.2019-07-25T23-32-26Z\nVary: Origin\nX-Amz-Request-Id: 15B4CB6A0BF922B1\nX-Xss-Protection: 1; mode=block\nDate: Thu, 25 Jul 2019 23:50:27 GMT\n}\n >>> [io.minio.MinioClient.executeReq(MinioClient.java:1204), io.minio.MinioClient.execute(MinioClient.java:1070), io.minio.MinioClient.executeDelete(MinioClient.java:1347), io.minio.MinioClient.abortMultipartUpload(MinioClient.java:5149), io.minio.MinioClient.putObject(MinioClient.java:4337), io.minio.MinioClient.putObject(MinioClient.java:4183), FunctionalTest.listIncompleteUploads_test2(FunctionalTest.java:1639), FunctionalTest.runTests(FunctionalTest.java:2989), FunctionalTest.main(FunctionalTest.java:3118)]"
}

Executed 9 out of 14 tests successfully.