Ignore ObjectNotFound errors in delete api while enforcing locking #11114

poornas · 2020-12-15T18:21:01Z

AWS does not report this or version not found as errors in the response.

Description

Motivation and Context

How to test this PR?

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

Fixes a regression (If yes, please add commit-id or PR # here)
Documentation needed
Unit tests needed

AWS does not report this or version not found as errors in the response.

kannappanr

LGTM

poornas · 2020-12-15T18:24:09Z

AWS does not report this or version not found as errors in the response.

Description

Motivation and Context

How to test this PR?

aws s3api delete-objects with non existent version on a object lock enabled bucket

Types of changes

Bug fix (non-breaking change which fixes an issue)

New feature (non-breaking change which adds functionality)

Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

Fixes a regression (If yes, please add commit-id or PR # here)

Documentation needed

Unit tests needed

minio-trusted · 2020-12-15T21:12:08Z

Mint Automation

Test	Result
mint-large-bucket.sh	✔️
mint-fs.sh	✔️
mint-gateway-s3.sh	✔️
mint-erasure.sh	✔️
mint-dist-erasure.sh	✔️
mint-zoned.sh	✔️
mint-gateway-nas.sh	✔️
mint-gateway-azure.sh	❌ more...

11114-c1944e4/mint-gateway-azure.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev6.minio.io:32454
ACCESS_KEY:           minioazure
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 29f5582bc606:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 9 seconds
(2/15) Running aws-sdk-java tests ... done in 1 seconds
(3/15) Running aws-sdk-php tests ... done in 2 minutes and 42 seconds
(4/15) Running aws-sdk-ruby tests ... done in 20 seconds
(5/15) Running awscli tests ... done in 2 minutes and 47 seconds
(6/15) Running healthcheck tests ... done in 0 seconds
(7/15) Running mc tests ... done in 3 minutes and 24 seconds
(8/15) Running minio-dotnet tests ... done in 1 minutes and 39 seconds
(9/15) Running minio-go tests ... FAILED in 1 minutes and 46 seconds
{
  "args": {
    "bucketName": "minio-go-test-a62kkvpvibrhuyfr",
    "objectName": "test-object",
    "opts": "",
    "size": -1
  },
  "duration": 1152,
  "function": "PutObject(bucketName, objectName, reader,size,opts)",
  "message": "Unexpected size",
  "name": "minio-go: testPutObjectStreaming",
  "status": "FAIL"
}
(9/15) Running minio-java tests ... FAILED in 8 minutes and 52 seconds
{
  "name": "minio-java",
  "function": "putObject()",
  "args": "[user metadata]",
  "duration": 169,
  "status": "FAIL",
  "error": "error occurred\nErrorResponse(code = AuthenticationFailed, message = -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.10.0/azblob/zc_storage_error.go:42\n===== RESPONSE ERROR (ServiceCode=AuthenticationFailed) =====\nDescription=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:403c7f93-a01e-0075-1223-d36a55000000\nTime:2020-12-15T20:49:39.6087176Z, Details: \n   AuthenticationErrorDetail: The MAC signature found in the HTTP request 'WhAqm5WTHS4r2rrhkmjfwCtUUPeH0AL9Sw6EKGx5kAU=' is not the same as any computed signature. Server used following string to sign: 'PUT\n\n\n128\n\napplication/xml\n\n\n\n\n\n\nx-ms-blob-cache-control:\nx-ms-blob-content-disposition:\nx-ms-blob-content-encoding:\nx-ms-blob-content-language:\nx-ms-blob-content-type:application/octet-stream\nx-ms-client-request-id:96e9b0af-980d-4b81-7a30-1395d89a2608\nx-ms-date:Tue, 15 Dec 2020 20:49:39 GMT\nx-ms-meta-my_header1:a   b   c\nx-ms-meta-my_header2:\"a   b   c\"\nx-ms-meta-my_project:Project One\nx-ms-meta-my_unicode_tag:å•†å“�\nx-ms-version:2019-02-02\n/minioazure/minio-java-test-3ijtmc6/minio-java-test-3esaoca\ncomp:blocklist\ntimeout:1501'.\n   Code: AuthenticationFailed\n   PUT https://minioazure.blob.core.windows.net/minio-java-test-3ijtmc6/minio-java-test-3esaoca?comp=blocklist&timeout=1501\n   Authorization: REDACTED\n   Content-Length: [128]\n   Content-Type: [application/xml]\n   User-Agent: [APN/1.0 MinIO/1.0 MinIO/2020-12-15T20:21:56Z]\n   X-Ms-Blob-Cache-Control: []\n   X-Ms-Blob-Content-Disposition: []\n   X-Ms-Blob-Content-Encoding: []\n   X-Ms-Blob-Content-Language: []\n   X-Ms-Blob-Content-Type: [application/octet-stream]\n   X-Ms-Client-Request-Id: [96e9b0af-980d-4b81-7a30-1395d89a2608]\n   X-Ms-Date: [Tue, 15 Dec 2020 20:49:39 GMT]\n   X-Ms-Meta-My_header1: [a   b   c]\n   X-Ms-Meta-My_header2: [\"a   b   c\"]\n   X-Ms-Meta-My_project: [Project One]\n   X-Ms-Meta-My_unicode_tag: [商品]\n   X-Ms-Version: [2019-02-02]\n   --------------------------------------------------------------------------------\n   RESPONSE Status: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\n   Content-Length: [1092]\n   Content-Type: [application/xml]\n   Date: [Tue, 15 Dec 2020 20:49:38 GMT]\n   Server: [Microsoft-HTTPAPI/2.0]\n   X-Ms-Error-Code: [AuthenticationFailed]\n   X-Ms-Request-Id: [403c7f93-a01e-0075-1223-d36a55000000]\n\n\n, bucketName = minio-java-test-3ijtmc6, objectName = minio-java-test-3esaoca, resource = /minio-java-test-3ijtmc6/minio-java-test-3esaoca, requestId = 1650FEF15867EDC1, hostId = 4e1e21c5-570b-43ea-848d-09846a543895)\nrequest={method=PUT, url=http://minio-dev6.minio.io:32454/minio-java-test-3ijtmc6/minio-java-test-3esaoca, headers=x-amz-meta-My-Unicode-Tag: 商品\nx-amz-meta-My-Project: Project One\nx-amz-meta-My-header1: a   b   c\nx-amz-meta-My-Header2: \"a   b   c\"\nContent-Type: application/octet-stream\nHost: minio-dev6.minio.io:32454\nAccept-Encoding: identity\nUser-Agent: MinIO (Linux; amd64) minio-java/8.0.3\nContent-MD5: A9oFTxee7YVcJ9fWsgQeKg==\nx-amz-content-sha256: 1ff7959f86334ddc5c188a5083268f600146328b2b6c5185e75bf7d9387d6b74\nx-amz-date: 20201215T204939Z\nAuthorization: AWS4-HMAC-SHA256 Credential=*REDACTED*/20201215/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date;x-amz-meta-my-header1;x-amz-meta-my-header2;x-amz-meta-my-project;x-amz-meta-my-unicode-tag, Signature=*REDACTED*\n}\nresponse={code=403, headers=Accept-Ranges: bytes\nContent-Length: 3086\nContent-Security-Policy: block-all-mixed-content\nContent-Type: application/xml\nServer: MinIO\nVary: Origin\nX-Amz-Request-Id: 1650FEF15867EDC1\nX-Xss-Protection: 1; mode=block\nDate: Tue, 15 Dec 2020 20:49:39 GMT\n}\n >>> [io.minio.MinioClient.execute(MinioClient.java:775), io.minio.MinioClient.putObject(MinioClient.java:4547), io.minio.MinioClient.putObject(MinioClient.java:2713), io.minio.MinioClient.putObject(MinioClient.java:2830), FunctionalTest.testPutObject(FunctionalTest.java:763), FunctionalTest.putObject(FunctionalTest.java:890), FunctionalTest.runObjectTests(FunctionalTest.java:3751), FunctionalTest.runTests(FunctionalTest.java:3783), FunctionalTest.main(FunctionalTest.java:3927)]"
}
(9/15) Running minio-js tests ... FAILED in 47 seconds
{
  "name": "minio-js",
  "function": "\"after all\" hook in \"functional tests\"",
  "duration": 79,
  "status": "FAIL",
  "error": "S3Error: The bucket you tried to delete is not empty at Object.parseError (node_modules/minio/dist/main/xml-parsers.js:68:11) at /mint/run/core/minio-js/node_modules/minio/dist/main/transformers.js:156:22 at DestroyableTransform._flush (node_modules/minio/dist/main/transformers.js:80:10) at DestroyableTransform.prefinish (node_modules/readable-stream/lib/_stream_transform.js:129:10) at prefinish (node_modules/readable-stream/lib/_stream_writable.js:611:14) at finishMaybe (node_modules/readable-stream/lib/_stream_writable.js:620:5) at endWritable (node_modules/readable-stream/lib/_stream_writable.js:643:3) at DestroyableTransform.Writable.end (node_modules/readable-stream/lib/_stream_writable.js:571:22) at IncomingMessage.onend (_stream_readable.js:684:10) at endReadableNT (_stream_readable.js:1327:12) at processTicksAndRejections (internal/process/task_queues.js:80:21)"
}
(9/15) Running minio-py tests ... done in 18 minutes and 13 seconds
(10/15) Running s3cmd tests ... done in 2 minutes and 16 seconds
(11/15) Running s3select tests ... done in 45 seconds
(12/15) Running security tests ... done in 0 seconds

Executed 12 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally

Ignore ObjectNotFound errors in delete api while enforcing locking

c1944e4

AWS does not report this or version not found as errors in the response.

poornas force-pushed the dllock branch from 1f346ad to c1944e4 Compare December 15, 2020 18:21

poornas changed the title ~~Filter ObjectNotFound errors in delete api while enforcing locking~~ Ignore ObjectNotFound errors in delete api while enforcing locking Dec 15, 2020

kannappanr approved these changes Dec 15, 2020

View reviewed changes

kannappanr requested a review from harshavardhana December 15, 2020 18:23

kannappanr requested a review from vadmeste December 15, 2020 18:43

kannappanr added the priority: high label Dec 15, 2020

harshavardhana approved these changes Dec 15, 2020

View reviewed changes

harshavardhana merged commit 3456b03 into minio:master Dec 15, 2020

poornas deleted the dllock branch June 13, 2022 22:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ignore ObjectNotFound errors in delete api while enforcing locking #11114

Ignore ObjectNotFound errors in delete api while enforcing locking #11114

poornas commented Dec 15, 2020 •

edited

kannappanr left a comment

poornas commented Dec 15, 2020

Description

Motivation and Context

How to test this PR?

Types of changes

Checklist:

minio-trusted commented Dec 15, 2020

Ignore ObjectNotFound errors in delete api while enforcing locking #11114

Ignore ObjectNotFound errors in delete api while enforcing locking #11114

Conversation

poornas commented Dec 15, 2020 • edited

Description

Motivation and Context

How to test this PR?

Types of changes

Checklist:

kannappanr left a comment

Choose a reason for hiding this comment

poornas commented Dec 15, 2020

Description

Motivation and Context

How to test this PR?

Types of changes

Checklist:

minio-trusted commented Dec 15, 2020

Mint Automation

11114-c1944e4/mint-gateway-azure.sh.log:

poornas commented Dec 15, 2020 •

edited