mc return error when applying an acl policy on RadosGW #2393

mldmld68 · 2018-02-15T15:53:51Z

Expected behaviour

No error while doing
./mc policy public service/bucket --insecure
on jewels and luminous Rados Gateway

Actual behaviour

On jewels :
mc: Unable to set policy public for service/bucket. invalid character '<' looking for beginning of value
=> ACL is NOT modified
On luminous :
mc: Unable to set policy public for service/bucket. 200 OK
=> ACL is modified even with this error message
#./mc --insecure policy service/bucket
Access permission for service/bucket is public

Steps to reproduce the behaviour

EP=https://myserver
BUCKET=mybucket
EPMCNICKNAME=mybucketnickname
EP=https://myserver

mc rm --recursive --force --insecure $EPMCNICKNAME/$BUCKET
mc mb $EPMCNICKNAME/$BUCKET --insecure
mc policy $EPMCNICKNAME/$BUCKET --insecure # Get bucket policy
mc cp test123.html $EPMCNICKNAME/$BUCKET --insecure
curl -k $EP/$BUCKET/test123.html
mc policy public $EPMCNICKNAME/$BUCKET --insecure # Set bucket policy
mc policy $EPMCNICKNAME/$BUCKET --insecure # Get bucket policy
curl -k $EP/$BUCKET/test123.html

Output:

./s3lab.sh

Removing myepnickname/mybucket/test123.html.
Removing myepnickname/mybucket.
Bucket created successfully myepnickname/mybucket.
Access permission for myepnickname/mybucket is none
test123.html: 17 B / 17 B ?¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦? 100.00% 596 B/s 0s

AccessDeniedmybuckettx0000000000000000001c9-005a85a63a-377b-default377b-default-default

mc: Unable to set policy public for myepnickname/mybucket. 200 OK

Access permission for myepnickname/mybucket is public

test123

==> File test123.html is downloaded without authentication ! OK

./mc version

Version: 2018-02-09T23:07:36Z
Release-tag: RELEASE.2018-02-09T23-07-36Z
Commit-id: 3987f14

System information

Centos 7.4

The text was updated successfully, but these errors were encountered:

kannappanr · 2018-02-15T21:02:57Z

@mldmld68 RadosGW does not support bucket policies yet.

mldmld68 · 2018-02-16T08:15:52Z

@kannappanr
Hi,
I disagree, RadosGW support bucket policies in Luminous, but not in Jewels.
Here is a test I made with aws cli on Rados GW.

test123.html

test123

acl-ro.json
{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"AddPerm",
"Effect":"Allow",
"Principal": "",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::mld-repo4/"]
}
]
}
s3lab-2.sh
export AWS_ACCESS_KEY_ID=xxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxx

EP=https://myserveurrunningRadiosGW.
BUCKET=mld-repo4
EPMCNICKNAME=labos3repo
mc rm --recursive --force --insecure $EPMCNICKNAME/$BUCKET
aws s3api create-bucket --no-verify-ssl --endpoint-url $EP --bucket $BUCKET
aws s3 ls --no-verify-ssl --endpoint-url $EP s3://$BUCKET
aws s3 cp test123.html --no-verify-ssl --endpoint-url $EP s3://$BUCKET
curl -k $EP/$BUCKET/test123.html
aws s3api put-bucket-policy --no-verify-ssl --endpoint-url $EP --bucket $BUCKET --policy file://acl-ro.json
curl -k $EP/$BUCKET/test123.html

LUMINOUS
Removing labos3repo/mld-repo4/test123.html.
Removing labos3repo/mld-repo4.
/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
upload: ./test123.html to s3://mld-repo4/test123.html

AccessDeniedmld-repo4tx0000000000000000002bd-005a85c9b8-377b-default377b-default-default

/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)

test123

Now, I can access the file test123, no more access denied

JEWELS
Removing openstackv1repo/mybucket/test123.html.
Removing openstackv1repo/mybucket.
/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
upload: ./test123.html to s3://mybucket/test123.html

AccessDeniedmybuckettx00000000000000000334b-005a85ca60-2ac98e-default2ac98e-default-default/usr/local/aws/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html

InsecureRequestWarning)

An error occurred (InvalidArgument) when calling the PutBucketPolicy operation: Unknown

AccessDeniedmybuckettx00000000000000000302c-005a85ca62-2c02a9-default2c02a9-default-default[

Now, I can't access the file test123, because the put-bucket failed

harshavardhana · 2018-02-21T07:03:22Z

I disagree, RadosGW support bucket policies in Luminous, but not in Jewels.

i do not see http://docs.ceph.com/docs/master/radosgw/s3/ bucket policies being supported are you sure? @mldmld68

harshavardhana · 2018-02-21T07:05:32Z

AccessDeniedmybuckettx0000000000000000001c9-005a85a63a-377b-default377b-default-default
mc: Unable to set policy public for myepnickname/mybucket. 200 OK

this error is returned by the server @mldmld68 i am not sure why it would say 200 OK and access denied. would it be possible for you to send a --debug output?

mldmld68 · 2018-02-21T09:46:41Z

@harshavardhana
Hi
On http://docs.ceph.com/docs/master/radosgw/s3/, one can read

But on http://docs.ceph.com/docs/luminous/radosgw/s3/

Feature	Status	Remarks
List Buckets	Supported
<...>
Policy (Buckets, Objects)	Not Supported	ACLs are supported

I'm testing on luminous, not master.
So it's not supported yet on luminous, but it works.

mldmld68 · 2018-02-21T09:49:45Z

Debug output of ./mc policy --debug --insecure public $EPNICKNAME/$BUCKET

mc: GET /mld-repo2/?location= HTTP/1.1
Host: myhost.mydomain
User-Agent: Minio (linux; amd64) minio-go/4.0.7 mc/2018-02-09T23:07:36Z
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20180221/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=REDACTED
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20180221T094853Z
Accept-Encoding: gzip

mc: HTTP/1.1 200 OK
Content-Length: 127
Date: Wed, 21 Feb 2018 09:45:52 GMT
X-Amz-Request-Id: tx00000000000000000005f-005a8d3fd0-377d-default

mc: TLS Certificate found:
mc: >> Country: FR
mc: >> Organization: XXXXXX
mc: >> Expires: 2020-10-30 17:03:25 +0000 UTC
mc: Response Time: 53.630574ms

mc: GET /mld-repo2/?policy= HTTP/1.1
Host: myhost.mydomain
User-Agent: Minio (linux; amd64) minio-go/4.0.7 mc/2018-02-09T23:07:36Z
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20180221/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=REDACTED
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20180221T094853Z
Accept-Encoding: gzip

mc: HTTP/1.1 404 Not Found
Content-Length: 276
Accept-Ranges: bytes
Content-Type: application/xml
Date: Wed, 21 Feb 2018 09:45:52 GMT
X-Amz-Request-Id: tx000000000000000000060-005a8d3fd0-377d-default

NoSuchBucketPolicyThe bucket policy does not existmld-repo2tx000000000000000000060-005a8d3fd0-377d-default377d-default-defaultmc: TLS Certificate found:

mc: >> Country: FR
mc: >> Organization: XXXXXXX
mc: >> Expires: 2020-10-30 17:03:25 +0000 UTC
mc: Response Time: 100.82246ms

mc: PUT /mld-repo2/?policy= HTTP/1.1
Host: myserver.mydomain
User-Agent: Minio (linux; amd64) minio-go/4.0.7 mc/2018-02-09T23:07:36Z
Content-Length: 421
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20180221/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date, Signature=REDACTED
Content-Md5: c+ktVvUZZtmjNbRYvOENAg==
X-Amz-Content-Sha256: 352ee63cdd46d98af454f9118e7139c2563d19acc193df0a5993b2a1bbd0d87a
X-Amz-Date: 20180221T094853Z
Accept-Encoding: gzip

mc: Content-Length: 0
Date: Wed, 21 Feb 2018 09:45:53 GMT
X-Amz-Request-Id: tx000000000000000000061-005a8d3fd1-377d-default

mc: TLS Certificate found:
mc: >> Country: FR
mc: >> Organization: XXXXXXX
mc: >> Expires: 2020-10-30 17:03:25 +0000 UTC
mc: Response Time: 18.961354ms

mc: Unable to set policy public for labos3repo/mld-repo2. 200 OK
(2) policy-main.go:356 cmd.runPolicyCmd(..) Tags: [labos3repo/mld-repo2, public]
(1) policy-main.go:216 cmd.doSetAccess(..) Tags: [labos3repo/mld-repo2, public]
(0) client-s3.go:805 cmd.(*s3Client).SetAccess(..)
Release-Tag:RELEASE.2018-02-09T23-07-36Z | Commit:3987f1405aa7 | Host:m320-mld | OS:linux | Arch:amd64 | Lang:go1.9.4 | Mem:2.6MB/5.2MB | Heap:2.6MB/3.7MB

kannappanr · 2018-02-21T19:45:38Z

@mldmld68 Thanks for the output.

If you look at the output,

mc: Unable to set policy public for labos3repo/mld-repo2. 200 OK

Ceph is returning 200 as output of put bucket policy. Please take a look at
https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTpolicy.html

If you look at the sample response, you can see that the returned status is 204

Ceph should be returning 204 instead of 200

kannappanr · 2018-02-26T22:31:32Z

@mldmld68 Closing this issue as answered. Please feel to reopen if you feel otherwise.

lock · 2020-04-25T13:55:19Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

harshavardhana assigned kannappanr Feb 21, 2018

kannappanr closed this as completed Feb 26, 2018

kannappanr added the out of scope label Feb 26, 2018

lock bot locked as resolved and limited conversation to collaborators Apr 25, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mc return error when applying an acl policy on RadosGW #2393

mc return error when applying an acl policy on RadosGW #2393

mldmld68 commented Feb 15, 2018 •

edited

kannappanr commented Feb 15, 2018

mldmld68 commented Feb 16, 2018 •

edited by kannappanr

harshavardhana commented Feb 21, 2018

harshavardhana commented Feb 21, 2018

mldmld68 commented Feb 21, 2018 •

edited

mldmld68 commented Feb 21, 2018 •

edited

kannappanr commented Feb 21, 2018 •

edited

kannappanr commented Feb 26, 2018

lock bot commented Apr 25, 2020

mc return error when applying an acl policy on RadosGW #2393

mc return error when applying an acl policy on RadosGW #2393

Comments

mldmld68 commented Feb 15, 2018 • edited

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

./s3lab.sh

test123

./mc version

System information

kannappanr commented Feb 15, 2018

mldmld68 commented Feb 16, 2018 • edited by kannappanr

test123

test123

harshavardhana commented Feb 21, 2018

harshavardhana commented Feb 21, 2018

mldmld68 commented Feb 21, 2018 • edited

mldmld68 commented Feb 21, 2018 • edited

kannappanr commented Feb 21, 2018 • edited

kannappanr commented Feb 26, 2018

lock bot commented Apr 25, 2020

mldmld68 commented Feb 15, 2018 •

edited

mldmld68 commented Feb 16, 2018 •

edited by kannappanr

mldmld68 commented Feb 21, 2018 •

edited

mldmld68 commented Feb 21, 2018 •

edited

kannappanr commented Feb 21, 2018 •

edited