Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow policy claim values to be optional in STS #10078

Merged
merged 1 commit into from
Jul 19, 2020
Merged

allow policy claim values to be optional in STS #10078

merged 1 commit into from
Jul 19, 2020

Conversation

harshavardhana
Copy link
Member

Description

allow claims to be optional in STS

Motivation and Context

not all claims need to be present for
the JWT claim, let the policies not
exist and only apply which are present

this behavior complies with service
account behavior as well

How to test this PR?

Nothing special not all claims in incoming JWT
need to be present

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Documentation needed
  • Unit tests needed

@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-xl.sh ✔️
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-dist-xl.sh ✔️
mint-zoned.sh ✔️
mint-gateway-nas.sh ✔️
mint-gateway-s3.sh ❌ more...
mint-gateway-azure.sh ❌ more...

10078-1aaf308/mint-gateway-s3.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev3.minio.io:30815
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 9898fbeecc68:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 0 seconds
(2/15) Running aws-sdk-java tests ... done in 1 seconds
(3/15) Running aws-sdk-php tests ... done in 41 seconds
(4/15) Running aws-sdk-ruby tests ... done in 2 seconds
(5/15) Running awscli tests ... done in 2 minutes and 0 seconds
(6/15) Running healthcheck tests ... done in 0 seconds
(7/15) Running mc tests ... done in 0 seconds
(8/15) Running minio-dotnet tests ... done in 29 seconds
(9/15) Running minio-go tests ... done in 32 seconds
(10/15) Running minio-java tests ... done in 23 seconds
(11/15) Running minio-js tests ... done in 37 seconds
(12/15) Running minio-py tests ... done in 1 minutes and 5 seconds
(13/15) Running s3cmd tests ... FAILED in 55 seconds
{
  "name": "s3cmd",
  "duration": "2657",
  "function": "test_get_object_multipart",
  "status": "FAIL",
  "error": "\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n    An unexpected error has occurred.\n  Please try reproducing the error using\n  the latest s3cmd code from the git master\n  branch found at:\n    https://github.com/s3tools/s3cmd\n  and have a look at the known issues list:\n    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions\n  If the error persists, please report the\n  following lines (removing any private\n  info as necessary) to:\n   s3tools-bugs@lists.sourceforge.net\n\n\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n\nInvoked as: /usr/local/bin/s3cmd --config /tmp/.s3cmd-8265/s3cfg get s3://s3cmd-test-bucket-23277/s3cmd-test-object-11643 s3cmd-test-object-11643.downloaded\nProblem: <class 'ConnectionResetError: [Errno 104] Connection reset by peer\nS3cmd:   2.1.0\npython:   3.6.9 (default, Apr 18 2020, 01:56:04) \n[GCC 8.4.0]\nenvironment LANG=C.UTF-8\n\nTraceback (most recent call last):\n  File \"/usr/local/bin/s3cmd\", line 3121, in <module>\n    rc = main()\n  File \"/usr/local/bin/s3cmd\", line 3030, in main\n    rc = cmd_func(args)\n  File \"/usr/local/bin/s3cmd\", line 604, in cmd_object_get\n    response = s3.object_get(uri, dst_stream, destination, start_position = start_position, extra_label = seq_label)\n  File \"/usr/local/lib/python3.6/dist-packages/S3/S3.py\", line 726, in object_get\n    response = self.recv_file(request, stream, labels, start_position)\n  File \"/usr/local/lib/python3.6/dist-packages/S3/S3.py\", line 1733, in recv_file\n    data = http_response.read(this_chunk)\n  File \"/usr/lib/python3.6/http/client.py\", line 459, in read\n    n = self.readinto(b)\n  File \"/usr/lib/python3.6/http/client.py\", line 503, in readinto\n    n = self.fp.readinto(b)\n  File \"/usr/lib/python3.6/socket.py\", line 586, in readinto\n    return self._sock.recv_into(b)\nConnectionResetError: [Errno 104] Connection reset by peer\n\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n    An unexpected error has occurred.\n  Please try reproducing the error using\n  the latest s3cmd code from the git master\n  branch found at:\n    https://github.com/s3tools/s3cmd\n  and have a look at the known issues list:\n    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions\n  If the error persists, please report the\n  above lines (removing any private\n  info as necessary) to:\n   s3tools-bugs@lists.sourceforge.net\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
}
(13/15) Running s3select tests ... done in 3 seconds
(14/15) Running security tests ... done in 0 seconds

Executed 14 out of 15 tests successfully.

10078-1aaf308/mint-gateway-azure.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev7.minio.io:30629
ACCESS_KEY:           minioazure
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 0a508442c48b:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 8 seconds
(2/15) Running aws-sdk-java tests ... done in 1 seconds
(3/15) Running aws-sdk-php tests ... done in 1 minutes and 31 seconds
(4/15) Running aws-sdk-ruby tests ... done in 15 seconds
(5/15) Running awscli tests ... done in 2 minutes and 36 seconds
(6/15) Running healthcheck tests ... done in 0 seconds
(7/15) Running mc tests ... done in 1 seconds
(8/15) Running minio-dotnet tests ... done in 1 minutes and 37 seconds
(9/15) Running minio-go tests ... done in 4 minutes and 44 seconds
(10/15) Running minio-java tests ... FAILED in 5 minutes and 25 seconds
{
  "name": "minio-java",
  "function": "listIncompleteUploads(ListIncompleteUploadsArgs args)",
  "args": "prefix: minio, recursive: true",
  "duration": 1528,
  "status": "FAIL",
  "error": "java.net.SocketException: Connection reset >>> [java.base/java.net.SocketInputStream.read(SocketInputStream.java:186), java.base/java.net.SocketInputStream.read(SocketInputStream.java:140), okio.Okio$2.read(Okio.java:140), okio.AsyncTimeout$2.read(AsyncTimeout.java:237), okio.RealBufferedSource.indexOf(RealBufferedSource.java:358), okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:230), okhttp3.internal.http1.Http1ExchangeCodec.readHeaderLine(Http1ExchangeCodec.java:242), okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:213), okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.java:115), okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:94), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142), okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117), okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117), okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142), okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142), okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117), okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229), okhttp3.RealCall.execute(RealCall.java:81), io.minio.MinioClient.execute(MinioClient.java:1014), io.minio.MinioClient.uploadPart(MinioClient.java:7781), io.minio.MinioClient.putObject(MinioClient.java:4706), io.minio.MinioClient.putObject(MinioClient.java:4881), FunctionalTest.listIncompleteUploads_test3(FunctionalTest.java:1640), FunctionalTest.runTests(FunctionalTest.java:3816), FunctionalTest.main(FunctionalTest.java:4056)]"
}
(10/15) Running minio-js tests ... done in 2 minutes and 32 seconds
(11/15) Running minio-py tests ... done in 9 minutes and 53 seconds
(12/15) Running s3cmd tests ... done in 1 minutes and 35 seconds
(13/15) Running s3select tests ... done in 1 minutes and 7 seconds
(14/15) Running security tests ... done in 0 seconds

Executed 14 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally

kannappanr
kannappanr approved these changes Jul 18, 2020
View reviewed changes
Copy link
Contributor

@kannappanr kannappanr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@harshavardhana
allow claims to be optional in STS
2d253d7 
not all claims need to be present for
the JWT claim, let the policies not
exist and only apply which are present
when generating the credentials

once credentials are generated then
those policies should exist, otherwise
the request will fail.
@harshavardhana harshavardhana changed the title allow claims to be optional in STS allow policy claim values to be optional in STS Jul 19, 2020
@harshavardhana harshavardhana merged commit 7764c54 into minio:master Jul 19, 2020
@harshavardhana harshavardhana deleted the claim branch July 19, 2020 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kannappanr kannappanr kannappanr approved these changes

@vadmeste vadmeste Awaiting requested review from vadmeste

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@harshavardhana @minio-trusted @kannappanr