Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: etcd IAM encryption fails due to incorrect kms.Context #12431

Merged
merged 1 commit into from
Jun 4, 2021
Merged

fix: etcd IAM encryption fails due to incorrect kms.Context #12431

merged 1 commit into from
Jun 4, 2021

Conversation

harshavardhana
Copy link
Member

Description

fix: etcd IAM encryption fails due to incorrect kms.Context

Motivation and Context

Due to incorrect KMS context constructed, we need to add
additional fallbacks and also fix the original root cause
to fix already migrated deployments.

Bonus remove double migration is avoided in gateway mode
for etcd, instead do it once in iam.Init(), also simplify
the migration by not migrating STS users instead let the
clients regenerate them.

How to test this PR?

  • Install etcd
  • Install release RELEASE.2021-04-22T15-44-28Z in gateway mode
  • Update release to RELEASE.2021-05-11T23-27-41Z in gateway mode, with value MINIO_KMS_SECRET_KEY set.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression yes all etcd deployments are affected
  • Documentation updated
  • Unit tests added/updated

@harshavardhana harshavardhana force-pushed the fix-bug branch 3 times, most recently from 82371aa to 48bc9f1 Compare June 4, 2021 02:20
@harshavardhana
fix: etcd IAM encryption fails due to incorrect kms.Context
b42d2db 
Due to incorrect KMS context constructed, we need to add
additional fallbacks and also fix the original root cause
to fix already migrated deployments.

Bonus remove double migration is avoided in gateway mode
for etcd, instead do it once in iam.Init(), also simplify
the migration by not migrating STS users instead let the
clients regenerate them.
@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-gateway-s3.sh ✔️
mint-erasure.sh ✔️
mint-dist-erasure.sh ✔️
mint-zoned.sh ✔️
mint-gateway-nas.sh ✔️
mint-compress-encrypt-dist-erasure.sh ❌ more...

12431-b42d2db/mint-compress-encrypt-dist-erasure.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev7.minio.io:32586
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 14fd0d3da78b:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 1 seconds
(2/15) Running aws-sdk-java tests ... done in 2 seconds
(3/15) Running aws-sdk-php tests ... done in 43 seconds
(4/15) Running aws-sdk-ruby tests ... done in 3 seconds
(5/15) Running awscli tests ... done in 2 minutes and 16 seconds
(6/15) Running healthcheck tests ... done in 0 seconds
(7/15) Running mc tests ... done in 1 minutes and 14 seconds
(8/15) Running minio-dotnet tests ... done in 41 seconds
(9/15) Running minio-go tests ... done in 2 minutes and 1 seconds
(10/15) Running minio-java tests ... FAILED in 1 minutes and 27 seconds
{
  "name": "minio-java",
  "function": "composeObject()",
  "args": "[single source with offset]",
  "duration": 72,
  "status": "FAIL",
  "error": "error occurred\nErrorResponse(code = InvalidArgument, message = Range specified is not valid for source object, bucketName = minio-java-test-3jnkas8, objectName = minio-java-test-354piuu, resource = /minio-java-test-3jnkas8/minio-java-test-354piuu, requestId = 168546436EAC0EC1, hostId = 9a032a60-2537-4fd6-913e-3ece8a0437ab)\nrequest={method=PUT, url=http://minio-dev7.minio.io:32586/minio-java-test-3jnkas8/minio-java-test-354piuu?uploadId=69e974f9-ca75-40e7-8659-1aa74240592e&partNumber=1, headers=x-amz-copy-source: /minio-java-test-3jnkas8/minio-java-test-2l37eag\nx-amz-copy-source-range: bytes=2048-1048575\nx-amz-copy-source-if-match: cb92d17a904ccec2e6e23b8bb66245fb\nHost: minio-dev7.minio.io:32586\nAccept-Encoding: identity\nUser-Agent: MinIO (Linux; amd64) minio-java/8.0.3\nContent-MD5: 1B2M2Y8AsgTpgAmY7PhCfg==\nx-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date: 20210604T042136Z\nAuthorization: AWS4-HMAC-SHA256 Credential=*REDACTED*/20210604/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-copy-source;x-amz-copy-source-if-match;x-amz-copy-source-range;x-amz-date, Signature=*REDACTED*\n}\nresponse={code=400, headers=Accept-Ranges: bytes\nContent-Length: 390\nContent-Security-Policy: block-all-mixed-content\nContent-Type: application/xml\nServer: MinIO\nVary: Origin\nX-Amz-Request-Id: 168546436EAC0EC1\nX-Xss-Protection: 1; mode=block\nDate: Fri, 04 Jun 2021 04:21:36 GMT\n}\n >>> [io.minio.MinioClient.execute(MinioClient.java:775), io.minio.MinioClient.uploadPartCopy(MinioClient.java:4804), io.minio.MinioClient.composeObject(MinioClient.java:1431), FunctionalTest.testComposeObject(FunctionalTest.java:2120), FunctionalTest.composeObjectTests(FunctionalTest.java:2145), FunctionalTest.composeObject(FunctionalTest.java:2300), FunctionalTest.runObjectTests(FunctionalTest.java:3758), FunctionalTest.runTests(FunctionalTest.java:3783), FunctionalTest.main(FunctionalTest.java:3927)]"
}
(10/15) Running minio-js tests ... done in 48 seconds
(11/15) Running minio-py tests ... done in 2 minutes and 45 seconds
(12/15) Running s3cmd tests ... done in 18 seconds
(13/15) Running s3select tests ... done in 7 seconds
(14/15) Running security tests ... done in 0 seconds

Executed 14 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally

vadmeste
vadmeste approved these changes Jun 4, 2021
View reviewed changes
Copy link
Member

@vadmeste vadmeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but not tested

cmd/config-encrypted.go Show resolved Hide resolved
kannappanr
kannappanr approved these changes Jun 4, 2021
View reviewed changes
Copy link
Contributor

@kannappanr kannappanr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested few scenarios. LGTM

@harshavardhana harshavardhana merged commit 36b2f6d into minio:master Jun 4, 2021
@harshavardhana harshavardhana deleted the fix-bug branch June 4, 2021 18:15
tristanessquare pushed a commit to iternity-dotcom/minio that referenced this pull request Sep 2, 2021
@harshavardhana @tristanessquare
fix: etcd IAM encryption fails due to incorrect kms.Context (minio#12431
a07037b 
)

Due to incorrect KMS context constructed, we need to add
additional fallbacks and also fix the original root cause
to fix already migrated deployments.

Bonus remove double migration is avoided in gateway mode
for etcd, instead do it once in iam.Init(), also simplify
the migration by not migrating STS users instead let the
clients regenerate them.
tristanessquare pushed a commit to iternity-dotcom/minio that referenced this pull request Oct 7, 2021
@harshavardhana @tristanessquare
fix: etcd IAM encryption fails due to incorrect kms.Context (minio#12431
1e6c76e 
)

Due to incorrect KMS context constructed, we need to add
additional fallbacks and also fix the original root cause
to fix already migrated deployments.

Bonus remove double migration is avoided in gateway mode
for etcd, instead do it once in iam.Init(), also simplify
the migration by not migrating STS users instead let the
clients regenerate them.
tristanessquare pushed a commit to iternity-dotcom/minio that referenced this pull request Oct 13, 2021
@harshavardhana @tristanessquare
fix: etcd IAM encryption fails due to incorrect kms.Context (minio#12431
a647008 
)

Due to incorrect KMS context constructed, we need to add
additional fallbacks and also fix the original root cause
to fix already migrated deployments.

Bonus remove double migration is avoided in gateway mode
for etcd, instead do it once in iam.Init(), also simplify
the migration by not migrating STS users instead let the
clients regenerate them.
rluetzner pushed a commit to iternity-dotcom/minio that referenced this pull request Oct 14, 2021
@harshavardhana @rluetzner
fix: etcd IAM encryption fails due to incorrect kms.Context (minio#12431
9c34666 
)

Due to incorrect KMS context constructed, we need to add
additional fallbacks and also fix the original root cause
to fix already migrated deployments.

Bonus remove double migration is avoided in gateway mode
for etcd, instead do it once in iam.Init(), also simplify
the migration by not migrating STS users instead let the
clients regenerate them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vadmeste vadmeste vadmeste approved these changes

@kannappanr kannappanr kannappanr approved these changes

@aead aead Awaiting requested review from aead

Assignees
No one assigned
Labels
priority: high regression (fixed)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@harshavardhana @minio-trusted @vadmeste @kannappanr