KMS Admin-API: add route and handler for KMS key info #7955

aead · 2019-07-19T14:06:31Z

Description

This commit adds an admin API route and handler for
requesting status information about a KMS key.

Therefore, the client specifies the KMS key ID (when
empty / not set the server takes the currently configured
default key-ID) and the server tries to perform a dummy encryption,
re-wrap and decryption operation. If all three succeed we know that
the server can access the KMS and has permissions to generate, re-wrap
and decrypt data keys (policy is set correctly).

Motivation and Context

KMS management

How to test this PR?

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

Fixes a regression (If yes, please add commit-id or PR # here)
Documentation needed
Unit tests needed
Functional tests needed (If yes, add mint PR # here: )

pkg/madmin/kms-commands.go

poornas

LGTM

aead · 2019-08-06T16:52:20Z

ping @krishnasrinivas

krishnasrinivas · 2019-08-06T18:04:55Z

@aead will check

pkg/madmin/kms-commands.go

aead · 2019-08-08T14:17:52Z

Added dummy UpdateKey operation. PTAL @poornas @krishnasrinivas

codecov-io · 2019-08-08T14:44:54Z

Codecov Report

❗ No coverage uploaded for pull request base (master@8a71b0e). Click here to learn what that means.
The diff coverage is 5.37%.

@@            Coverage Diff            @@
##             master    #7955   +/-   ##
=========================================
  Coverage          ?   44.38%           
=========================================
  Files             ?      323           
  Lines             ?    52713           
  Branches          ?        0           
=========================================
  Hits              ?    23396           
  Misses            ?    27142           
  Partials          ?     2175

Impacted Files	Coverage Δ
cmd/admin-handlers.go	`12.13% <0%> (ø)`
pkg/madmin/kms-commands.go	`0% <0%> (ø)`
cmd/admin-router.go	`100% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a71b0e...67f7561. Read the comment docs.

pkg/madmin/kms-commands.go

cmd/admin-handlers.go

sinhaashish · 2019-08-12T04:40:08Z

@aead : I still find the panic on the server

Object API (Amazon S3 compatible):
   Go:         https://docs.min.io/docs/golang-client-quickstart-guide
   Java:       https://docs.min.io/docs/java-client-quickstart-guide
   Python:     https://docs.min.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
2019/08/12 10:08:57 http: panic serving 127.0.0.1:48182: interface conversion: interface {} is string, not []uint8
goroutine 121 [running]:
net/http.(*conn).serve.func1(0xc000548140)
	/usr/local/go/src/net/http/server.go:1769 +0x139
panic(0x17a3ae0, 0xc0004587e0)
	/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/minio/minio/cmd.criticalErrorHandler.ServeHTTP.func1(0x1cdba20, 0xc0002d07e0, 0xc00017ea00)
	/home/ashish/code/go/src/github.com/minio/minio/cmd/generic-handlers.go:801 +0x248
panic(0x17a3ae0, 0xc0004587e0)
	/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/minio/minio/cmd/crypto.(*vaultService).UpdateKey(0xc0003d8500, 0xc0000441c9, 0xc, 0xc00058ad20, 0x59, 0x60, 0xc00018de30, 0x5116d8499a9e5735, 0xc00058ad20, 0x59, ...)
	/home/ashish/code/go/src/github.com/minio/minio/cmd/crypto/vault.go:277 +0x41f
github.com/minio/minio/cmd.adminAPIHandlers.KMSKeyStatusHandler(0x1cd9760, 0xc00028aa60, 0xc00017ec00)
	/home/ashish/code/go/src/github.com/minio/minio/cmd/admin-handlers.go:1702 +0x5bd
net/http.HandlerFunc.ServeHTTP(...)
	/usr/local/go/src/net/http/server.go:1995
github.com/minio/minio/cmd.httpTraceAll.func1(0x1cd9760, 0xc00028aa60, 0xc00017ec00)
	/home/ashish/code/go/src/github.com/minio/minio/cmd/handler-utils.go:331 +0x13b
net/http.HandlerFunc.ServeHTTP(0xc0004a57e0, 0x1cd9760, 0xc00028aa60, 0xc00017ec00)
	/usr/local/go/src/net/http/server.go:1995 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc000180240, 0x1cd9760, 0xc00028aa60, 0xc00017ea00)

aead · 2019-08-12T08:27:04Z

@sinhaashish Will rebase the PR. Have you rebased on master? Since it works for me...

sinhaashish · 2019-08-12T10:17:45Z

@sinhaashish Will rebase the PR. Have you rebased on master? Since it works for me...

Thanks , It works now.

aead · 2019-08-26T09:58:33Z

Ping @krishnasrinivas @poornas

poornas · 2019-08-26T18:03:22Z

@aead, build failures on this

aead · 2019-08-26T18:08:27Z

@poornas Not related to the PR - see tcp 127.0.0.1:9009: connect: connection refused Caused by starting server in erasure mode.

poornas · 2019-08-26T19:08:46Z

@aead, can you add this API to madmin docs, otherwise looks good.

pkg/madmin/examples/kms-status.go

harshavardhana · 2019-09-04T01:01:41Z

please resolve conflicts @aead

This commit adds an admin API route and handler for requesting status information about a KMS key. Therefore, the client specifies the KMS key ID (when empty / not set the server takes the currently configured default key-ID) and the server tries to perform a dummy encryption, re-wrap and decryption operation. If all three succeed we know that the server can access the KMS and has permissions to generate, re-wrap and decrypt data keys (policy is set correctly). Co-Authored-By: poornas <poornas@users.noreply.github.com>

Co-Authored-By: poornas <poornas@users.noreply.github.com>

minio-ops · 2019-09-04T09:51:40Z

Mint Automation

Test	Result
mint-compression-xl.sh	✔️
mint-xl.sh	✔️
mint-compression-dist-xl.sh	✔️
mint-compression-fs.sh	✔️
mint-worm.sh	✔️
mint-fs.sh	✔️
mint-dist-xl.sh	✔️
mint-gateway-nas.sh	✔️
mint-large-bucket.sh	❌ more...

7955-67f7561/mint-large-bucket.sh.log:

Running with
SERVER_ENDPOINT:      72.28.97.60:31979
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 0a4be878a74d:/mint/log /tmp/mint-logs'

(1/14) Running aws-sdk-go tests ... done in 2 seconds
(2/14) Running aws-sdk-java tests ... done in 2 seconds
(3/14) Running aws-sdk-php tests ... done in 49 seconds
(4/14) Running aws-sdk-ruby tests ... done in 15 seconds
(5/14) Running awscli tests ... done in 2 minutes and 36 seconds
(6/14) Running healthcheck tests ... done in 0 seconds
(7/14) Running mc tests ... done in 1 minutes and 14 seconds
(8/14) Running minio-dotnet tests ... done in 3 minutes and 22 seconds
(9/14) Running minio-go tests ... FAILED in 8 minutes and 21 seconds
{
  "args": {},
  "duration": 49230,
  "error": "The specified key does not exist.",
  "function": "testStorageClassMetadataCopyObject()",
  "message": "Stat failed",
  "name": "minio-go: testStorageClassMetadataCopyObject",
  "status": "FAIL"
}

Executed 8 out of 14 tests successfully.

Error: No such image: minio/minio:7955-67f7561

aead requested review from krishnasrinivas and poornas July 19, 2019 14:06

poornas requested changes Jul 31, 2019

View reviewed changes

pkg/madmin/kms-commands.go Show resolved Hide resolved

aead force-pushed the kms-admin-api branch from 2868bca to d0eadbd Compare July 31, 2019 20:55

poornas approved these changes Jul 31, 2019

View reviewed changes

aead force-pushed the kms-admin-api branch from d0eadbd to 9b82d57 Compare August 1, 2019 07:29

krishnasrinivas reviewed Aug 6, 2019

View reviewed changes

pkg/madmin/kms-commands.go Outdated Show resolved Hide resolved

aead mentioned this pull request Aug 7, 2019

KMS Admin-API: add route and handler for KMS key listing #8033

Closed

7 tasks

aead force-pushed the kms-admin-api branch 2 times, most recently from b7123ca to 8ac2e8e Compare August 8, 2019 14:16

aead requested a review from poornas August 8, 2019 14:17

poornas requested changes Aug 8, 2019

View reviewed changes

pkg/madmin/kms-commands.go Show resolved Hide resolved

cmd/admin-handlers.go Outdated Show resolved Hide resolved

aead force-pushed the kms-admin-api branch from 354d681 to 0b0fdc3 Compare August 9, 2019 07:14

aead mentioned this pull request Aug 10, 2019

fix type conversion in UpdateKey for Vault #8058

Merged

7 tasks

aead force-pushed the kms-admin-api branch from 0b0fdc3 to 9642dbb Compare August 12, 2019 08:27

aead force-pushed the kms-admin-api branch from 9642dbb to 848d746 Compare August 12, 2019 15:54

aead requested review from krishnasrinivas and poornas August 19, 2019 15:30

aead force-pushed the kms-admin-api branch from 848d746 to c1a757b Compare August 26, 2019 18:09

aead force-pushed the kms-admin-api branch 3 times, most recently from 7c7e4cb to 4d8668a Compare August 28, 2019 15:29

poornas reviewed Aug 28, 2019

View reviewed changes

pkg/madmin/examples/kms-status.go Outdated Show resolved Hide resolved

poornas approved these changes Aug 28, 2019

View reviewed changes

aead force-pushed the kms-admin-api branch 2 times, most recently from 5b75f1f to 2df9fdd Compare August 29, 2019 10:32

krishnasrinivas approved these changes Sep 4, 2019

View reviewed changes

Andreas Auernhammer and others added 2 commits September 4, 2019 11:10

Update pkg/madmin/examples/kms-status.go

67f7561

Co-Authored-By: poornas <poornas@users.noreply.github.com>

aead force-pushed the kms-admin-api branch from 2df9fdd to 67f7561 Compare September 4, 2019 09:10

kannappanr merged commit 810a44e into minio:master Sep 4, 2019

aead mentioned this pull request Sep 23, 2019

KMS Admin-API add route and handler for KMS key rotation #8295

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KMS Admin-API: add route and handler for KMS key info #7955

KMS Admin-API: add route and handler for KMS key info #7955

aead commented Jul 19, 2019 •

edited

poornas left a comment

aead commented Aug 6, 2019

krishnasrinivas commented Aug 6, 2019

aead commented Aug 8, 2019

codecov-io commented Aug 8, 2019 •

edited

sinhaashish commented Aug 12, 2019

aead commented Aug 12, 2019

sinhaashish commented Aug 12, 2019

aead commented Aug 26, 2019

poornas commented Aug 26, 2019

aead commented Aug 26, 2019

poornas commented Aug 26, 2019

harshavardhana commented Sep 4, 2019

minio-ops commented Sep 4, 2019

KMS Admin-API: add route and handler for KMS key info #7955

KMS Admin-API: add route and handler for KMS key info #7955

Conversation

aead commented Jul 19, 2019 • edited

Description

Motivation and Context

How to test this PR?

Types of changes

Checklist:

poornas left a comment

Choose a reason for hiding this comment

aead commented Aug 6, 2019

krishnasrinivas commented Aug 6, 2019

aead commented Aug 8, 2019

codecov-io commented Aug 8, 2019 • edited

Codecov Report

sinhaashish commented Aug 12, 2019

aead commented Aug 12, 2019

sinhaashish commented Aug 12, 2019

aead commented Aug 26, 2019

poornas commented Aug 26, 2019

aead commented Aug 26, 2019

poornas commented Aug 26, 2019

harshavardhana commented Sep 4, 2019

minio-ops commented Sep 4, 2019

Mint Automation

7955-67f7561/mint-large-bucket.sh.log:

aead commented Jul 19, 2019 •

edited

codecov-io commented Aug 8, 2019 •

edited