New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rgw: respect Swift's negative, HTTP referer-based ACL grants. #14344

Merged
merged 2 commits into from Jun 13, 2017

Conversation

Projects
None yet
3 participants
@rzarzynski
Contributor

rzarzynski commented Apr 5, 2017

For the sake of simplicity and compatibility with S3's ACLs, this patch doesn't handle the case of having multiple .r:* in an ACL like:

  .r:*,.r:-.example.com,.r:*

.r:* is handled specifically because of S3. In the future we can get full support by parsing the whole acl grant map.

Signed-off-by: Radoslaw Zarzynski rzarzynski@mirantis.com

rzarzynski added some commits Apr 5, 2017

rgw: partially respect Swift's negative, HTTP referer-based ACLs.
For the sake of simplicity this patch doesn't handle the case of
having multiple ".r:*" in a single ACL like:

  .r:*,.r:-.example.com,.r:*

The global wildcard (.r:*) is handled specifically because of S3.
Next patch will brings full support.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
rgw: fully respect Swift's negative, HTTP referer-based ACLs.
Fixes: http://tracker.ceph.com/issues/18841
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Apr 5, 2017

Just pushed a full support for Swift's negative, referrer-based ACLs. Both s3-tests and Tempest are happy with these patches. Manual testing verified their operability:

rgw$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:*,.r:-.example.com,.r:*"
HTTP/1.1 204 No Content
X-Trans-Id: tx0000000000000000002fa-0058e540e8-1024-default
Content-Type: text/plain; charset=utf-8
Date: Wed, 05 Apr 2017 19:09:28 GMT


rgw$ curl -i "${publicURL}/cont/aaa10" -X GET -H "Referer: https://dev.example.com/en-US/docs/Web/JavaScript"
HTTP/1.1 200 OK
Content-Length: 3
Accept-Ranges: bytes
Last-Modified: Wed, 05 Apr 2017 19:08:26 GMT
X-Timestamp: 1491419306.44163
etag: 202cb962ac59075b964b07152d234b70
X-Trans-Id: tx0000000000000000002fb-0058e540fc-1024-default
Content-Type: application/x-www-form-urlencoded
Date: Wed, 05 Apr 2017 19:09:48 GMT

123
rgw$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:*,.r:-.example.com"
HTTP/1.1 204 No Content
X-Trans-Id: tx0000000000000000002fc-0058e5410b-1024-default
Content-Type: text/plain; charset=utf-8
Date: Wed, 05 Apr 2017 19:10:03 GMT


rgw$ curl -i "${publicURL}/cont/aaa10" -X GET -H "Referer: https://dev.example.com/en-US/docs/Web/JavaScript"
HTTP/1.1 401 Unauthorized
Content-Length: 12
X-Trans-Id: tx0000000000000000002fd-0058e54114-1024-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Wed, 05 Apr 2017 19:10:12 GMT

AccessDenied
rgw$ curl -i "${publicURL}/cont/aaa10" -X GET -H "Referer: https://dev.mozilla.com/en-US/docs/Web/JavaScript"  
HTTP/1.1 200 OK
Content-Length: 3
Accept-Ranges: bytes
Last-Modified: Wed, 05 Apr 2017 19:08:26 GMT
X-Timestamp: 1491419306.44163
etag: 202cb962ac59075b964b07152d234b70
X-Trans-Id: tx0000000000000000002fe-0058e54121-1024-default
Content-Type: application/x-www-form-urlencoded
Date: Wed, 05 Apr 2017 19:10:25 GMT

123

CC: @mattbenjamin, @mdw-at-linuxbox, @Jing-Scott.

@oritwas

This comment has been minimized.

Contributor

oritwas commented Apr 6, 2017

lgtm. Did you run it against tempest?

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Apr 6, 2017

@oritwas: yes, I verified the branch with Tempest yesterday. It hasn't found any regression in comparison to master (39349a4) in the whole tempest.api.object_storage campaign.

Anyway, I double checked today for the two ACL-related tests affected in #13294:

$ ./run_tempest.sh -V tempest.api.object_storage.test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds tempest.api.object_storage.test_object_services.PublicObjectTest.test_access_public_object_with_another_user_creds
WARNING: This script is deprecated and will be removed in the near future. Please migrate to tempest run or another method of launching a test runner
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list 
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover}  --load-list /tmp/tmpxl5tLj
{0} tempest.api.object_storage.test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds [0.183404s] ... ok
{0} tempest.api.object_storage.test_object_services.PublicObjectTest.test_access_public_object_with_another_user_creds [0.261094s] ... ok

======
Totals
======
Ran: 2 tests in 3.0000 sec.
 - Passed: 2
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 0.4445 sec.

==============
Worker Balance
==============
 - Worker 0 (2 tests) => 0:00:00.445107
* based ACLs. We need to go through all items to respect negative
* grants. */
uint32_t referer_perm = current_perm;
for (const auto& r : referer_list) {

This comment has been minimized.

@Jing-Scott

Jing-Scott Apr 6, 2017

Contributor

I have a question that if set .r:-.exmple.com,.r:*.example.com on container read acl, the www.example.com have the access in SWIFT and in this change without finding from the rear of referer_list, is it worked as expected?

This comment has been minimized.

@rzarzynski

rzarzynski Apr 6, 2017

Contributor

The case of .r:-.example.com,.r:*.example.com:

  • RadosGW:
rgw$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:-.example.com,.r:*.example.com"
HTTP/1.1 204 No Content
X-Trans-Id: tx00000000000000000000b-0058e644b2-1024-default
Content-Type: text/plain; charset=utf-8
Date: Thu, 06 Apr 2017 13:37:54 GMT

rgw$ curl -i "${publicURL}/cont/aaa10" -X GET -H "Referer: https://www.example.com"
HTTP/1.1 200 OK
Content-Length: 3
Accept-Ranges: bytes
Last-Modified: Thu, 06 Apr 2017 13:37:03 GMT
X-Timestamp: 1491485823.30361
etag: 202cb962ac59075b964b07152d234b70
X-Trans-Id: tx00000000000000000000c-0058e64512-1024-default
Content-Type: application/x-www-form-urlencoded
Date: Thu, 06 Apr 2017 13:39:30 GMT

123
  • Swift:
swift$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:-.example.com,.r:*.example.com"
HTTP/1.1 204 No Content
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txb9790199c6684ab9931f6-0058e64524
Date: Thu, 06 Apr 2017 13:39:48 GMT

swift$ curl -i "${publicURL}/cont/aaa10" -X GET -H "Referer: https://www.example.com"
HTTP/1.1 200 OK
Content-Length: 3
Accept-Ranges: bytes
Last-Modified: Wed, 25 Jan 2017 18:50:32 GMT
Etag: 202cb962ac59075b964b07152d234b70
X-Timestamp: 1485370231.32458
Content-Type: application/x-www-form-urlencoded
X-Trans-Id: txfe59ab709d004f5ca6acf-0058e6452e
Date: Thu, 06 Apr 2017 13:39:58 GMT

123

The case of .r:-.exmple.com,.r:*.example.com (original, without a in the first item):

  • RadosGW:
rgw$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:-.exmple.com,.r:*.example.com"
HTTP/1.1 204 No Content
X-Trans-Id: tx00000000000000000000d-0058e6460e-1024-default
Content-Type: text/plain; charset=utf-8
Date: Thu, 06 Apr 2017 13:43:42 GMT

rgw$ curl -i "${publicURL}/cont/aaa10" -X GET -H "Referer: https://www.example.com"
HTTP/1.1 200 OK
Content-Length: 3
Accept-Ranges: bytes
Last-Modified: Thu, 06 Apr 2017 13:37:03 GMT
X-Timestamp: 1491485823.30361
etag: 202cb962ac59075b964b07152d234b70
X-Trans-Id: tx00000000000000000000e-0058e64616-1024-default
Content-Type: application/x-www-form-urlencoded
Date: Thu, 06 Apr 2017 13:43:50 GMT

123
  • Swift:
swift$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:-.exmple.com,.r:*.example.com"
HTTP/1.1 204 No Content
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txf89be419de33481191fea-0058e64637
Date: Thu, 06 Apr 2017 13:44:23 GMT

swift$ curl -i "${publicURL}/cont/aaa10" -X GET -H "Referer: https://www.example.com"
HTTP/1.1 200 OK
Content-Length: 3
Accept-Ranges: bytes
Last-Modified: Wed, 25 Jan 2017 18:50:32 GMT
Etag: 202cb962ac59075b964b07152d234b70
X-Timestamp: 1485370231.32458
Content-Type: application/x-www-form-urlencoded
X-Trans-Id: tx3ea6d1f8b6fe4e4fbdbfd-0058e6463a
Date: Thu, 06 Apr 2017 13:44:26 GMT

123

This comment has been minimized.

@Jing-Scott

Jing-Scott Apr 6, 2017

Contributor

sorry, this is a typo. i've lost a. :-)

This comment has been minimized.

@Jing-Scott

Jing-Scott Apr 6, 2017

Contributor

lgtm, thanks!

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Apr 6, 2017

@yehudasa: here are results of the manual verification of the S3 and Swift's negative ACLs intersection:

rgw-s3$ curl -i "http://127.0.0.1:8000/cont/aaa10" -X GET -H "Referer: https://www.example.com"
HTTP/1.1 403 Forbidden
Content-Length: 214
x-amz-request-id: tx00000000000000000000b-0058e67b96-1027-default
Accept-Ranges: bytes
Content-Type: application/xml
Date: Thu, 06 Apr 2017 17:32:06 GMT

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><BucketName>cont</BucketName><RequestId>tx00000000000000000000b-0058e67b96-1027-default</RequestId><HostId>1027-default-default</HostId></Error>

rgw-swift$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:*"
HTTP/1.1 204 No Content
X-Trans-Id: tx00000000000000000000c-0058e67bcc-1027-default
Content-Type: text/plain; charset=utf-8
Date: Thu, 06 Apr 2017 17:33:01 GMT

rgw-s3$ curl -i "http://127.0.0.1:8000/cont/aaa10" -X GET -H "Referer: https://www.example.com"
HTTP/1.1 200 OK
Content-Length: 3
Accept-Ranges: bytes
Last-Modified: Thu, 06 Apr 2017 17:30:52 GMT
ETag: "202cb962ac59075b964b07152d234b70"
x-amz-request-id: tx00000000000000000000d-0058e67bd6-1027-default
Content-Type: application/x-www-form-urlencoded
Date: Thu, 06 Apr 2017 17:33:10 GMT

123
rgw-swift$ curl -i "${publicURL}/cont" -X POST -H "X-Auth-Token: ${token}"  -H "X-Container-Read: .r:*,.r:-.example.com"
HTTP/1.1 204 No Content
X-Trans-Id: tx00000000000000000000e-0058e67c09-1027-default
Content-Type: text/plain; charset=utf-8
Date: Thu, 06 Apr 2017 17:34:01 GMT

rgw-s3$ curl -i "http://127.0.0.1:8000/cont/aaa10" -X GET -H "Referer: https://www.example.com"
HTTP/1.1 403 Forbidden
Content-Length: 214
x-amz-request-id: tx00000000000000000000f-0058e67c0d-1027-default
Accept-Ranges: bytes
Content-Type: application/xml
Date: Thu, 06 Apr 2017 17:34:05 GMT

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><BucketName>cont</BucketName><RequestId>tx00000000000000000000f-0058e67c0d-1027-default</RequestId><HostId>1027-default-default</HostId></Error>

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented May 1, 2017

@yehudasa: this branch has been tested in following Teuthology runs:

The results look good. The failures have been caused by Valgrind signalising a non-related leak outside of RadosGW.

@oritwas

oritwas approved these changes May 1, 2017

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jun 13, 2017

jenkins retest this please

@oritwas oritwas merged commit 51b512a into ceph:master Jun 13, 2017

5 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
Unmodified Submodules submodules for project are unmodified
Details
default Build finished.
Details
make check make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment