Check bucket & prefix after obj's for new bucket are fetched #7838

kaankabalak · 2019-06-26T21:43:44Z

Description

When the request to list objects in a bucket concludes, a check has been introduced to see if the current bucket name and prefix are the same as when the request was made before updating the displayed objects.

Motivation and Context

Fixes #7837

Regression

Yes, from #7805

How Has This Been Tested?

Create a bucket named large with 100k objects.
Open the Network tab on the Browser Dev Console to track requests
Create a bucket named small with a single object.
Click on the bucket name large from the sidebar to load its contents.
While the request is still going on (when we see the spinner and before the objects are loaded), click on the bucket name small to load its contents.
Observe that the objects corresponding to bucket named small are displayed instantly as the request to retrieve them doesn't take long.
Track the request to list the objects for the bucket named large and verify that the request has ended and the objects corresponding to that bucket are not loaded on top.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added unit tests to cover my changes.
I have added/updated functional tests in mint. (If yes, add mint PR # here: )
All new and existing tests passed.

Fixes minio#7837

kanagarajkm

Verified.

codecov · 2019-06-27T18:23:12Z

Codecov Report

Merging #7838 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #7838      +/-   ##
==========================================
- Coverage   46.09%   46.07%   -0.02%     
==========================================
  Files         299      299              
  Lines       48644    48644              
==========================================
- Hits        22424    22415       -9     
- Misses      24135    24145      +10     
+ Partials     2085     2084       -1

Impacted Files	Coverage Δ
cmd/retry.go	`82.14% <0%> (-5.36%)`	⬇️
cmd/xl-sets.go	`46.99% <0%> (-1.02%)`	⬇️
pkg/certs/certs.go	`58.76% <0%> (+4.12%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3f9f8b...e8fa22b. Read the comment docs.

minio-ops · 2019-06-27T18:48:28Z

Mint Automation

Test	Result
mint-compression-xl.sh	✔️
mint-compression-dist-xl.sh	✔️
mint-worm.sh	✔️
mint-fs.sh	✔️
mint-gateway-nas.sh	✔️
mint-xl.sh	❌ more...
mint-large-bucket.sh	❌ more...
mint-compression-fs.sh	❌ more...
mint-dist-xl.sh	❌ more...

7838-e8fa22b/mint-xl.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.57:30408
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp d92a8a305905:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... FAILED in 4 seconds
{
  "alert": "",
  "args": {
    "bucketName": "aws-sdk-go-test-6yf2gw11zsejth",
    "expiry": 60000000000,
    "objectName": "presignedTest"
  },
  "duration": 3060,
  "error": "RequestError: send request failed\ncaused by: Put http://72.28.97.57:30408/aws-sdk-go-test-6yf2gw11zsejth: dial tcp 72.28.97.57:30408: connect: no route to host",
  "function": "PresignedPut",
  "message": "AWS SDK Go CreateBucket Failed",
  "name": "aws-sdk-go",
  "status": "FAIL"
}

Executed 0 out of 14 tests successfully.

7838-e8fa22b/mint-dist-xl.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.60:30082
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 2145c69fc1e2:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... done in 0 seconds
(2/14) Running aws-sdk-java tests ... done in 1 seconds
(3/14) Running aws-sdk-php tests ... done in 48 seconds
(4/14) Running aws-sdk-ruby tests ... done in 3 seconds
(5/14) Running awscli tests ... done in 1 minutes and 54 seconds
(6/14) Running healthcheck tests ... done in 1 seconds
(7/14) Running mc tests ... done in 39 seconds
(8/14) Running minio-dotnet tests ... done in 33 seconds
(9/14) Running minio-go tests ... done in 1 minutes and 0 seconds
(10/14) Running minio-java tests ... FAILED in 1 minutes and 8 seconds
{
  "name": "minio-java",
  "function": "removeIncompleteUpload(String bucketName, String objectName)",
  "duration": 945,
  "status": "FAIL",
  "error": "error occurred\nErrorResponse(code=SlowDown, message=Please reduce your request, bucketName=minio-java-test-1bnp3uh, objectName=minio-java-test-10tvbjb, resource=/minio-java-test-1bnp3uh/minio-java-test-10tvbjb, requestId=15AC22857FFD6A44, hostId=ea3494ef-d5b5-4cdb-bbe9-453f4887daed)\nrequest={method=DELETE, url=http://72.28.97.60:30082/minio-java-test-1bnp3uh/minio-java-test-10tvbjb?uploadId=9fa75a03-5b91-4dd3-b757-1d43601fa69d, headers=Host: 72.28.97.60:30082\nUser-Agent: MinIO (amd64; amd64) minio-java/dev\nx-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date: 20190627T184527Z\nAuthorization: AWS4-HMAC-SHA256 Credential=*REDACTED*/20190627/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=*REDACTED*\n}\nresponse={code=503, headers=Accept-Ranges: bytes\nContent-Length: 363\nContent-Security-Policy: block-all-mixed-content\nContent-Type: application/xml\nRetry-After: 120\nServer: MinIO/DEVELOPMENT.2019-06-27T18-26-22Z\nVary: Origin\nX-Amz-Request-Id: 15AC22857FFD6A44\nX-Minio-Deployment-Id: ea3494ef-d5b5-4cdb-bbe9-453f4887daed\nX-Xss-Protection: 1; mode=block\nDate: Thu, 27 Jun 2019 18:45:28 GMT\n}\n >>> [io.minio.MinioClient.executeReq(MinioClient.java:1204), io.minio.MinioClient.execute(MinioClient.java:1070), io.minio.MinioClient.executeDelete(MinioClient.java:1347), io.minio.MinioClient.abortMultipartUpload(MinioClient.java:5149), io.minio.MinioClient.putObject(MinioClient.java:4337), io.minio.MinioClient.putObject(MinioClient.java:4183), FunctionalTest.removeIncompleteUploads_test(FunctionalTest.java:1719), FunctionalTest.runTests(FunctionalTest.java:2992), FunctionalTest.main(FunctionalTest.java:3118)]"
}

Executed 9 out of 14 tests successfully.

7838-e8fa22b/mint-large-bucket.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.57:31238
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 7589e41d8570:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... FAILED in 3 seconds
{
  "alert": "",
  "args": {
    "bucketName": "aws-sdk-go-test-mrcmxdubslw3p5",
    "expiry": 60000000000,
    "objectName": "presignedTest"
  },
  "duration": 2485,
  "error": "RequestError: send request failed\ncaused by: Put http://72.28.97.57:31238/aws-sdk-go-test-mrcmxdubslw3p5: dial tcp 72.28.97.57:31238: connect: no route to host",
  "function": "PresignedPut",
  "message": "AWS SDK Go CreateBucket Failed",
  "name": "aws-sdk-go",
  "status": "FAIL"
}

Executed 0 out of 14 tests successfully.

7838-e8fa22b/mint-compression-fs.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.57:30409
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 52ef4b6d3218:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... FAILED in 3 seconds
{
  "alert": "",
  "args": {
    "bucketName": "aws-sdk-go-test-3d504xqeogidi9",
    "expiry": 60000000000,
    "objectName": "presignedTest"
  },
  "duration": 2569,
  "error": "RequestError: send request failed\ncaused by: Put http://72.28.97.57:30409/aws-sdk-go-test-3d504xqeogidi9: dial tcp 72.28.97.57:30409: connect: no route to host",
  "function": "PresignedPut",
  "message": "AWS SDK Go CreateBucket Failed",
  "name": "aws-sdk-go",
  "status": "FAIL"
}

Executed 0 out of 14 tests successfully.

Check bucket & prefix after obj's for new bucket are fetched

0ad9aed

Fixes minio#7837

kannappanr requested review from kanagarajkm and krishnasrinivas June 26, 2019 22:15

kannappanr added the priority: high label Jun 26, 2019

kanagarajkm approved these changes Jun 27, 2019

View reviewed changes

krishnasrinivas approved these changes Jun 27, 2019

View reviewed changes

Merge branch 'master' into bucket-switch-bug

e8fa22b

kannappanr merged commit 36c19f1 into minio:master Jun 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check bucket & prefix after obj's for new bucket are fetched #7838

Check bucket & prefix after obj's for new bucket are fetched #7838

kaankabalak commented Jun 26, 2019

kanagarajkm left a comment

codecov bot commented Jun 27, 2019 •

edited

minio-ops commented Jun 27, 2019

Check bucket & prefix after obj's for new bucket are fetched #7838

Check bucket & prefix after obj's for new bucket are fetched #7838

Conversation

kaankabalak commented Jun 26, 2019

Description

Motivation and Context

Regression

How Has This Been Tested?

Types of changes

Checklist:

kanagarajkm left a comment

Choose a reason for hiding this comment

codecov bot commented Jun 27, 2019 • edited

Codecov Report

minio-ops commented Jun 27, 2019

Mint Automation

7838-e8fa22b/mint-xl.sh.log:

7838-e8fa22b/mint-dist-xl.sh.log:

7838-e8fa22b/mint-large-bucket.sh.log:

7838-e8fa22b/mint-compression-fs.sh.log:

codecov bot commented Jun 27, 2019 •

edited