New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rgw: add support for Swift's TempURLs with prefix-based scope #16370

Merged
merged 3 commits into from Aug 2, 2017

Conversation

Projects
None yet
3 participants
@rzarzynski
Contributor

rzarzynski commented Jul 17, 2017

No description provided.

rgw: add the get_optional() method to RGWHTTPArgs.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
std::string string_concat_reserve(const Args&... args)
{
return string_join_reserve(boost::string_view{}, args...);
}

This comment has been minimized.

@cbodley

cbodley Jul 17, 2017

Contributor

see string_cat_reserve() above

This comment has been minimized.

@rzarzynski

rzarzynski Jul 17, 2017

Contributor

I've completely missed it out. Thanks, @cbodley!

rgw: switch from boost::string_ref to boost::string_view in rgw_swift…
…_auth.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
if (prefix) {
const auto prefix_uri = string_cat_reserve(no_obj_uri, *prefix);
is_auth_ok = boost::algorithm::starts_with(decoded_uri, prefix_uri);
}

This comment has been minimized.

@cbodley

cbodley Jul 18, 2017

Contributor

i'm having trouble following this logic. does this prefix case really not depend on the input string rhs?

rgw: add support for Swift's TempURLs with prefix-based scope.
Fixes: http://tracker.ceph.com/issues/20398
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
@cbodley

looks good. bonus points if we're able to run against upstream swift's TestTempURLPrefix

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jul 28, 2017

@cbodley: after fixing unrelated issues in TempURL (#16636, #16658, #16659), the TestTempURLPrefix became green:

(swift) [rzarzynski@rzarzynski functional]$ SWIFT_TEST_CONFIG_FILE=rgw.conf nosetests -v test.functional.test_tempurl:TestTempURLPrefix
SKIPPING FUNCTIONAL TESTS SPECIFIC TO AUTH VERSION 3
SKIPPING FUNCTIONAL TESTS SPECIFIC TO SERVICE TOKENS
SKIPPING FUNCTIONAL TESTS DUE TO NO CONFIG FOR RESELLER ADMIN
SKIPPING FUNCTIONAL TESTS SPECIFIC TO AUTH VERSION 3
SKIPPING FUNCTIONAL TESTS SPECIFIC TO SERVICE TOKENS
SKIPPING FUNCTIONAL TESTS DUE TO NO CONFIG FOR RESELLER ADMIN
test_GET (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_GET_DLO_inside_container (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_GET_DLO_outside_container (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_GET_with_key_2 (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_HEAD (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_PUT (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_PUT_manifest_access (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_changing_expires (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_changing_sig (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_different_object (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_empty_prefix (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_missing_query_parm (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_no_prefix_match (test.functional.test_tempurl.TestTempURLPrefix) ... ok
test_object_url_with_prefix (test.functional.test_tempurl.TestTempURLPrefix) ... ok

----------------------------------------------------------------------
Ran 14 tests in 5.403s

OK

@rzarzynski rzarzynski removed the DNM label Jul 28, 2017

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jul 28, 2017

@cbodley: I'm setting following configurables:

rgw_swift_enforce_content_length=true
rgw_keystone_implicit_tenants=true
rgw_keystone_accepted_admin_roles "admin"
rgw_swift_versioning_enabled=true
# especially following ones are crucial
rgw_swift_account_in_url=true
rgw_swift_auth_entry="swift/auth" 
rgw frontends = civetweb port=8000 prefix=/swift

Also we need to workaround the lack of frontend's prefix support in TempAUTH as the whole Swift API must be placed in the root of URL hierarchy:

diff --git a/src/rgw/rgw_swift_auth.cc b/src/rgw/rgw_swift_auth.cc
index e7a25f5..e64c73a 100644
--- a/src/rgw/rgw_swift_auth.cc
+++ b/src/rgw/rgw_swift_auth.cc
@@ -695,7 +695,7 @@ void RGW_SWIFT_Auth_Get::execute()
     tenant_path.append(info.user_id.to_str());
   }
 
-  dump_header(s, "X-Storage-Url", swift_url + swift_prefix + "/v1" +
+  dump_header(s, "X-Storage-Url", swift_url + "/v1" +
               tenant_path);
 
   using rgw::auth::swift::encode_token;

The rgw.conf config file for Swift's functional tests:

[func_test]
# sample config
auth_host = 127.0.0.1
auth_port = 8000
auth_ssl = no
auth_prefix = /auth/

# Primary functional test account (needs admin access to the account)
account = test
username = tester
password = testing

# User on a second account (needs admin access to the account)
account2 = test2
username2 = tester2
password2 = testing2

# User on same account as first, but without admin access
username3 = tester3
password3 = testing3

collate = C

A script for creating users:

#!/bin/bash
set -e

# FIXME:
#  * tester3
#  * tenanted accounts

if [ "${PWD##*/}" == "build" ]
then
  BINDIR="bin"
else
  BINDIR="."
fi

# The first credential for both manual testing and the Functional
# Tests of Swift
${BINDIR}/radosgw-admin user create                             \
                --uid="test"                                    \
                --display-name="John Doe"                       \
                --email="john@example.com"                      &&
${BINDIR}/radosgw-admin subuser create                          \
                --uid="test"                                    \
                --subuser="test:tester"                         \
                --access=full                                   &&
${BINDIR}/radosgw-admin key create                              \
                --subuser="test:tester"                         \
                --key-type=swift                                \
                --secret="testing"

# The second credentuals solely for the functional tests
${BINDIR}/radosgw-admin user create                             \
                --uid="test2"                                   \
                --display-name="John Doe 2"                     \
                --email="john2@example.com"                     &&
${BINDIR}/radosgw-admin subuser create                          \
                --uid="test2"                                   \
                --subuser="test2:tester2"                       \
                --access=full                                   &&
${BINDIR}/radosgw-admin key create                              \
                --subuser="test2:tester2"                       \
                --key-type=swift                                \
                --secret="testing2"

@yuriw yuriw merged commit f6adc5f into ceph:master Aug 2, 2017

4 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
make check (arm64) make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment