Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gcs: use MD5Sum as ETag if present in object attrs #7643

Merged
merged 1 commit into from
May 16, 2019

Conversation

poornas
Copy link
Contributor

@poornas poornas commented May 14, 2019

Fixes: 7642

Description

for single PUT object call on GCS gateway - use the MD5 object attribute if set. Otherwise use the CRC32c hash as it is guaranteed to be present.

Motivation and Context

This would allow md5 compatibility in --compat mode

Regression

No

How Has This Been Tested?

see repro steps in issue

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added unit tests to cover my changes.
  • I have added/updated functional tests in mint. (If yes, add mint PR # here: )
  • All new and existing tests passed.

@minio-ops
Copy link

Mint Automation

Test Result
mint-compression-xl.sh ✔️
mint-compression-dist-xl.sh ✔️
mint-compression-fs.sh ✔️
mint-worm.sh ✔️
mint-fs.sh ✔️
mint-dist-xl.sh ✔️
mint-gateway-nas.sh ✔️
mint-xl.sh more...
mint-large-bucket.sh more...

7643-74732f3/mint-xl.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.61:30068
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp ebb582a47e05:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... FAILED in 6 minutes and 0 seconds
{
  "alert": "",
  "args": {
    "bucketName": "aws-sdk-go-test-gnssbs1mbskacs",
    "expiry": 60000000000,
    "objectName": "presignedTest"
  },
  "duration": 360010,
  "error": "XMinioServerNotInitialized: Server not initialized, please try again.\n\tstatus code: 503, request id: 159EA0BF8CB34385, host id: ",
  "function": "PresignedPut",
  "message": "AWS SDK Go CreateBucket Failed",
  "name": "aws-sdk-go",
  "status": "FAIL"
}

Executed 0 out of 14 tests successfully.

7643-74732f3/mint-large-bucket.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.61:31395
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp a2d0eca434ac:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... done in 2 seconds
(2/14) Running aws-sdk-java tests ... done in 1 seconds
(3/14) Running aws-sdk-php tests ... done in 50 seconds
(4/14) Running aws-sdk-ruby tests ... FAILED in 4 minutes and 10 seconds
{
  "name": "aws-sdk-ruby",
  "function": "copyObject(source_bucket_name, target_bucket_name, source_file_name, target_file_name = )",
  "args": {
    "source_bucket_name": "aws-sdk-ruby-bucket-363c79a3fe35",
    "target_bucket_name": "aws-sdk-ruby-bucket-f66d9b8bd146",
    "source_file_name": "datafile-1-kB",
    "target_file_name": "datafile-1-kB"
  },
  "duration": 242482.76,
  "error": "Net::ReadTimeout",
  "status": "FAIL"
}

Executed 3 out of 14 tests successfully.

@codecov
Copy link

codecov bot commented May 14, 2019

Codecov Report

Merging #7643 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7643      +/-   ##
==========================================
+ Coverage   47.27%   47.28%   +0.01%     
==========================================
  Files         294      294              
  Lines       47172    47176       +4     
==========================================
+ Hits        22300    22308       +8     
+ Misses      22799    22796       -3     
+ Partials     2073     2072       -1
Impacted Files Coverage Δ
cmd/gateway/gcs/gateway-gcs.go 22.19% <100%> (+0.29%) ⬆️
cmd/fs-v1.go 61.57% <0%> (-0.38%) ⬇️
cmd/posix.go 62.4% <0%> (+0.38%) ⬆️
cmd/bitrot-streaming.go 86.95% <0%> (+3.26%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c90a28...74732f3. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants