Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change SetRemoteTarget API to allow editing remote target #12175

Merged
merged 1 commit into from Apr 28, 2021

Conversation

poornas
Copy link
Contributor

@poornas poornas commented Apr 28, 2021

more flexibly.

  • Previously only credentials could be updated with
    mc admin bucket remote edit. Change this to allow updating
    synchronous replication flag, path, bandwidth and healthcheck

Add a flag to allow disabling proxying in active-active replication

Description

Motivation and Context

How to test this PR?

with minio/mc#3701

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Documentation updated
  • Unit tests added/updated

pkg/madmin/remote-target-commands.go Outdated Show resolved Hide resolved
pkg/madmin/remote-target-commands.go Outdated Show resolved Hide resolved
more flexibly.

- Previously only credentials could be updated with
`mc admin bucket remote edit`. Change this to allow updating
synchronous replication flag, path, bandwidth and healthcheck

Add a flag to allow disabling proxying in active-active replication
@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-gateway-s3.sh ✔️
mint-erasure.sh ✔️
mint-dist-erasure.sh ✔️
mint-gateway-nas.sh ✔️
mint-zoned.sh more...
mint-compress-encrypt-dist-erasure.sh more...

12175-ad28335/mint-zoned.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev7.minio.io:32178
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 73ab23510212:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 2 seconds
(2/15) Running aws-sdk-java tests ... done in 1 seconds
(3/15) Running aws-sdk-php tests ... done in 43 seconds
(4/15) Running aws-sdk-ruby tests ... done in 3 seconds
(5/15) Running awscli tests ... done in 2 minutes and 10 seconds
(6/15) Running healthcheck tests ... done in 1 seconds
(7/15) Running mc tests ... done in 1 minutes and 7 seconds
(8/15) Running minio-dotnet tests ... done in 53 seconds
(9/15) Running minio-go tests ... done in 2 minutes and 4 seconds
(10/15) Running minio-java tests ... done in 1 minutes and 50 seconds
(11/15) Running minio-js tests ... FAILED in 16 seconds
{
  "name": "minio-js",
  "function": "\"after all\" hook in \"functional tests\"",
  "duration": 5,
  "status": "FAIL",
  "error": "S3Error: The bucket you tried to delete is not empty at Object.parseError (node_modules/minio/dist/main/xml-parsers.js:79:11) at /mint/run/core/minio-js/node_modules/minio/dist/main/transformers.js:156:22 at DestroyableTransform._flush (node_modules/minio/dist/main/transformers.js:80:10) at DestroyableTransform.prefinish (node_modules/readable-stream/lib/_stream_transform.js:129:10) at prefinish (node_modules/readable-stream/lib/_stream_writable.js:611:14) at finishMaybe (node_modules/readable-stream/lib/_stream_writable.js:620:5) at endWritable (node_modules/readable-stream/lib/_stream_writable.js:643:3) at DestroyableTransform.Writable.end (node_modules/readable-stream/lib/_stream_writable.js:571:22) at IncomingMessage.onend (internal/streams/readable.js:684:10) at endReadableNT (internal/streams/readable.js:1327:12) at processTicksAndRejections (internal/process/task_queues.js:80:21)"
}
(11/15) Running minio-py tests ... FAILED in 2 minutes and 40 seconds
{
  "name": "minio-py:test_thread_safe",
  "status": "FAIL",
  "args": {
    "bucket_name": "minio-py-test-878ac29c-9590-4534-a0ed-809a3da7c4a6",
    "object_name": "ace320b8-f1ba-4d9c-bb61-ec98eab92da9"
  },
  "message": "S3 operation failed; code: BucketNotEmpty, message: The bucket you tried to delete is not empty, resource: /minio-py-test-878ac29c-9590-4534-a0ed-809a3da7c4a6, request_id: 167A180F20E7C86A, host_id: 4bcf3720-fbc9-43f1-92d4-c541b41bc3e4, bucket_name: minio-py-test-878ac29c-9590-4534-a0ed-809a3da7c4a6",
  "error": "Traceback (most recent call last):\n  File \"/mint/run/core/minio-py/tests.py\", line 126, in _call_test\n    func(log_entry, *args, **kwargs)\n  File \"/mint/run/core/minio-py/tests.py\", line 1640, in test_thread_safe\n    _CLIENT.remove_bucket(bucket_name)\n  File \"/usr/local/lib/python3.6/dist-packages/minio/api.py\", line 676, in remove_bucket\n    self._execute(\"DELETE\", bucket_name)\n  File \"/usr/local/lib/python3.6/dist-packages/minio/api.py\", line 409, in _execute\n    preload_content=preload_content,\n  File \"/usr/local/lib/python3.6/dist-packages/minio/api.py\", line 385, in _url_open\n    raise response_error\nminio.error.S3Error: S3 operation failed; code: BucketNotEmpty, message: The bucket you tried to delete is not empty, resource: /minio-py-test-878ac29c-9590-4534-a0ed-809a3da7c4a6, request_id: 167A180F20E7C86A, host_id: 4bcf3720-fbc9-43f1-92d4-c541b41bc3e4, bucket_name: minio-py-test-878ac29c-9590-4534-a0ed-809a3da7c4a6\n",
  "duration": 5933
}
(11/15) Running s3cmd tests ... done in 18 seconds
(12/15) Running s3select tests ... done in 6 seconds
(13/15) Running security tests ... done in 0 seconds

Executed 13 out of 15 tests successfully.

12175-ad28335/mint-compress-encrypt-dist-erasure.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev6.minio.io:32298
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 1610a51739c2:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 1 seconds
(2/15) Running aws-sdk-java tests ... done in 2 seconds
(3/15) Running aws-sdk-php tests ... done in 42 seconds
(4/15) Running aws-sdk-ruby tests ... done in 4 seconds
(5/15) Running awscli tests ... FAILED in 31 seconds
{
  "name": "awscli",
  "duration": 2709,
  "function": "aws --endpoint-url http://minio-dev6.minio.io:32298 s3api copy-object --bucket awscli-mint-test-bucket-2640 --key datafile-1-kB-copy --copy-source awscli-mint-test-bucket-2640/datafile-1-kB\n",
  "status": "FAIL",
  "error": "Hash mismatch expected 084e1383b70fb0c51acc680fef370023, got ac57de7156d7fc25ac1a65f81fa3989b"
}
(5/15) Running healthcheck tests ... done in 1 seconds
(6/15) Running mc tests ... done in 47 seconds
(7/15) Running minio-dotnet tests ... done in 41 seconds
(8/15) Running minio-go tests ... FAILED in 2 minutes and 43 seconds
{
  "args": {},
  "duration": 703,
  "error": "At least one of the pre-conditions you specified did not hold",
  "function": "CopyObjectPart(destination, source)",
  "message": "CopyObjectPart call failed",
  "name": "minio-go: testUnencryptedToSSES3CopyObjectPart",
  "status": "FAIL"
}
(8/15) Running minio-java tests ... FAILED in 2 minutes and 6 seconds
{
  "name": "minio-java",
  "function": "copyObject()",
  "args": "[match etag]",
  "duration": 129,
  "status": "FAIL",
  "error": "error occurred\nErrorResponse(code = PreconditionFailed, message = At least one of the pre-conditions you specified did not hold, bucketName = minio-java-test-3kn880m, objectName = minio-java-test-2ne718b-copy, resource = /minio-java-test-3kn880m/minio-java-test-2ne718b-copy, requestId = 167A17D3F1CC3059, hostId = 2f8fa777-0ef9-451f-b6d7-f92e5c2b84fa)\nrequest={method=PUT, url=http://minio-dev6.minio.io:32298/minio-java-test-3kn880m/minio-java-test-2ne718b-copy, headers=x-amz-copy-source-if-match: 71cff0a060f852067e443ad1e24ae26c-1\nx-amz-copy-source: /minio-java-test-hckhpu/minio-java-test-2ne718b\nHost: minio-dev6.minio.io:32298\nAccept-Encoding: identity\nUser-Agent: MinIO (Linux; amd64) minio-java/8.0.3\nContent-MD5: 1B2M2Y8AsgTpgAmY7PhCfg==\nx-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date: 20210428T180655Z\nAuthorization: AWS4-HMAC-SHA256 Credential=*REDACTED*/20210428/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-copy-source;x-amz-copy-source-if-match;x-amz-date, Signature=*REDACTED*\n}\nresponse={code=412, headers=Accept-Ranges: bytes\nContent-Length: 418\nContent-Security-Policy: block-all-mixed-content\nContent-Type: application/xml\nETag: \"71cff0a060f852067e443ad1e24ae26c\"\nLast-Modified: Wed, 28 Apr 2021 18:06:55 GMT\nServer: MinIO\nVary: Origin\nX-Amz-Request-Id: 167A17D3F1CC3059\nX-Xss-Protection: 1; mode=block\nDate: Wed, 28 Apr 2021 18:06:55 GMT\n}\n >>> [io.minio.MinioClient.execute(MinioClient.java:775), io.minio.MinioClient.execute(MinioClient.java:563), io.minio.MinioClient.executePut(MinioClient.java:904), io.minio.MinioClient.copyObject(MinioClient.java:1232), FunctionalTest.testCopyObjectMatchETag(FunctionalTest.java:1850), FunctionalTest.copyObject(FunctionalTest.java:2016), FunctionalTest.runObjectTests(FunctionalTest.java:3757), FunctionalTest.runTests(FunctionalTest.java:3783), FunctionalTest.main(FunctionalTest.java:3927)]"
}
(8/15) Running minio-js tests ... done in 51 seconds
(9/15) Running minio-py tests ... done in 2 minutes and 51 seconds
(10/15) Running s3cmd tests ... FAILED in 5 seconds
{
  "name": "s3cmd",
  "duration": "2793",
  "function": "test_put_object_multipart",
  "status": "FAIL",
  "error": "WARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Too many failures. Giving up on '/mint/data/datafile-65-MB'\nERROR: \nUpload of '/mint/data/datafile-65-MB' part 1 failed. Use\n  /usr/local/bin/s3cmd abortmp s3://s3cmd-test-bucket-10483/s3cmd-test-object-7383 73c8df83-2f90-4b48-8c1a-86945a4a7365\nto abort the upload, or\n  /usr/local/bin/s3cmd --upload-id 73c8df83-2f90-4b48-8c1a-86945a4a7365 put ...\nto continue the upload.\nERROR: Upload of '/mint/data/datafile-65-MB' failed too many times (Last reason: )"
}
(10/15) Running s3select tests ... done in 6 seconds
(11/15) Running security tests ... done in 0 seconds

Executed 11 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally

@poornas poornas requested a review from kannappanr April 28, 2021 19:25
@harshavardhana harshavardhana merged commit 632252f into minio:master Apr 28, 2021
@harshavardhana harshavardhana deleted the disableproxy branch April 28, 2021 22:26
@poornas poornas restored the disableproxy branch April 28, 2021 22:29
@poornas poornas deleted the disableproxy branch August 8, 2022 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants