New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rgw: make RGWEnv return a const ref. to its map #15269

Merged
merged 1 commit into from Jun 14, 2017

Conversation

Projects
None yet
4 participants
@theanalyst
Member

theanalyst commented May 24, 2017

We already have a public method set for setting values in the RGW env
map, making the map as a read only. In addition:

  • Added const to other methods in RGWEnv which are getters
  • req_info::init_meta also reused the same iter name for an internal
    find, changed the var name
  • add_grants_headers now accepts an RGWEnv instead of the map
  • use range based for loops wherever the code was changed\
  • req_info holds a ptr to a const RGWEnv, since we seem to only read the
    values from it

Signed-off-by: Abhishek Lekshmanan abhishek@suse.com

@theanalyst theanalyst requested review from rzarzynski and cbodley May 24, 2017

@rzarzynski

Thanks for bringing this up, @theanalyst! Please see my comments below.

@@ -201,7 +201,7 @@ static string get_abs_path(const string& request_uri) {
return request_uri.substr(beg_pos, len - beg_pos);
}
req_info::req_info(CephContext *cct, class RGWEnv *e) : env(e) {
req_info::req_info(CephContext *cct, class RGWEnv *_env) : env(_env) {

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

The underscore in _env may be omitted:

req_info::req_info(CephContext *cct, class RGWEnv *env) : env(env) {

A bit more const-correct variant would look like:

req_info::req_info(CephContext* const cct, class RGWEnv* const env) : env(env) {
map<string, string, ltstr_nocase>& m = env->get_map();
map<string, string, ltstr_nocase>::iterator iter;
for (iter = m.begin(); iter != m.end(); ++iter) {
for (const auto& iter: env->get_map()) {

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

Hmm, technically it isn't an iterator anymore. Maybe just kv instead?

This comment has been minimized.

@theanalyst

This comment has been minimized.

@theanalyst
@@ -427,8 +424,8 @@ void req_info::init_meta_info(bool *found_bad_meta)
}
}
}
for (iter = x_meta_map.begin(); iter != x_meta_map.end(); ++iter) {
dout(10) << "x>> " << iter->first << ":" << rgw::crypt_sanitize::x_meta_map{iter->first, iter->second} << dendl;
for (const auto& iter: x_meta_map) {

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

The same as above (I stop mentioning that).

This comment has been minimized.

@theanalyst

This comment has been minimized.

@theanalyst
@@ -383,16 +383,16 @@ class RGWEnv {
void init(CephContext *cct);
void init(CephContext *cct, char **envp);
void set(const boost::string_ref& name, const boost::string_ref& val);
const char *get(const char *name, const char *def_val = NULL);
int get_int(const char *name, int def_val = 0);
const char *get(const char *name, const char *def_val = NULL) const;

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

As we're already touching the line it might be worth to switch to nullptr as well.

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

hm sure

string old = iter->second;
auto it = x_meta_map.find(name_low);
if (it != x_meta_map.end()) {
string old = it->second;
int pos = old.find_last_not_of(" \t"); /* get rid of any whitespaces after the value */

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

May we have size_t in place of int?

This comment has been minimized.

@theanalyst
headers.push_back(pair<string, string>(iter->first, iter->second));
const auto & m = new_env.get_map();
for (const auto& iter: m) {
headers.push_back(pair<string, string>(iter.first, iter.second));

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

std::make_pair could be helpful here.

This comment has been minimized.

@cbodley

cbodley May 30, 2017

Contributor

headers.emplace_back(iter) works here as well

This comment has been minimized.

@theanalyst
string old = iter->second;
auto it = x_meta_map.find(name_low);
if (it != x_meta_map.end()) {
string old = it->second;
int pos = old.find_last_not_of(" \t"); /* get rid of any whitespaces after the value */
old = old.substr(0, pos + 1);

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

I know it's not directly related to the proposed changes but we might have an issue here. AFAIK std::string::npos usually is the max of size_t, so adding 1 to it would give us 0. If so, the substr(0, 0) eradicates the old value. This isn't coherent with the comment above if my understanding is correct.

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

checking for pos != string::npos (or use boost::string::trim_right..)

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

I vote for trim_right. We're already using boost::algorithm::trim in couple of places.

This comment has been minimized.

@theanalyst
@@ -201,7 +201,7 @@ static string get_abs_path(const string& request_uri) {
return request_uri.substr(beg_pos, len - beg_pos);
}
req_info::req_info(CephContext *cct, class RGWEnv *e) : env(e) {
req_info::req_info(CephContext *cct, class RGWEnv *_env) : env(_env) {

This comment has been minimized.

@rzarzynski

rzarzynski May 30, 2017

Contributor

Just a suggestion: maybe req_info::env could be turned into a pointer to constant. If I see correctly the methods of req_info call only RGWEnv::get.

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

Yeah I thought of going that route, and got around to fixing all the places where we mutate the map without the methods. I also vote for this.

This comment has been minimized.

@theanalyst

@rzarzynski rzarzynski self-assigned this May 30, 2017

map<string, string, ltstr_nocase>& m = env->get_map();
map<string, string, ltstr_nocase>::iterator iter;
for (iter = m.begin(); iter != m.end(); ++iter) {
for (const auto& iter: env->get_map()) {

This comment has been minimized.

@theanalyst
string old = iter->second;
auto it = x_meta_map.find(name_low);
if (it != x_meta_map.end()) {
string old = it->second;
int pos = old.find_last_not_of(" \t"); /* get rid of any whitespaces after the value */

This comment has been minimized.

@theanalyst
for (iter = x_meta_map.begin(); iter != x_meta_map.end(); ++iter) {
dout(10) << "x>> " << iter->first << ":" << rgw::crypt_sanitize::x_meta_map{iter->first, iter->second} << dendl;
for (const auto& iter: x_meta_map) {
dout(10) << "x>> " << iter.first << ":" << rgw::crypt_sanitize::x_meta_map{iter.first, iter.second} << dendl;

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

also do the dout->ldout stuff while I'm here

This comment has been minimized.

@theanalyst

theanalyst Jun 1, 2017

Member

unfortunately don't have a context, I just looked at the code in github's ui which folded the code when I made the comment

string old = iter->second;
auto it = x_meta_map.find(name_low);
if (it != x_meta_map.end()) {
string old = it->second;
int pos = old.find_last_not_of(" \t"); /* get rid of any whitespaces after the value */
old = old.substr(0, pos + 1);

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

checking for pos != string::npos (or use boost::string::trim_right..)

@@ -427,8 +424,8 @@ void req_info::init_meta_info(bool *found_bad_meta)
}
}
}
for (iter = x_meta_map.begin(); iter != x_meta_map.end(); ++iter) {
dout(10) << "x>> " << iter->first << ":" << rgw::crypt_sanitize::x_meta_map{iter->first, iter->second} << dendl;
for (const auto& iter: x_meta_map) {

This comment has been minimized.

@theanalyst
@@ -383,16 +383,16 @@ class RGWEnv {
void init(CephContext *cct);
void init(CephContext *cct, char **envp);
void set(const boost::string_ref& name, const boost::string_ref& val);
const char *get(const char *name, const char *def_val = NULL);
int get_int(const char *name, int def_val = 0);
const char *get(const char *name, const char *def_val = NULL) const;

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

hm sure

const char *get(const char *name, const char *def_val = NULL);
int get_int(const char *name, int def_val = 0);
const char *get(const char *name, const char *def_val = NULL) const;
int get_int(const char *name, int def_val = 0) const;
bool get_bool(const char *name, bool def_val = 0);

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

this should also be const if we go for const ptr

headers.push_back(pair<string, string>(iter->first, iter->second));
const auto & m = new_env.get_map();
for (const auto& iter: m) {
headers.push_back(pair<string, string>(iter.first, iter.second));

This comment has been minimized.

@theanalyst
for (iter = m.begin(); iter != m.end(); ++iter) {
headers.push_back(pair<string, string>(iter->first, iter->second));
for (const auto& iter: new_env.get_map()) {
headers.push_back(pair<string, string>(iter.first, iter.second));

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

emplace

This comment has been minimized.

@theanalyst
/* add original headers that start with HTTP_X_AMZ_ */
#define SEARCH_AMZ_PREFIX "HTTP_X_AMZ_"
for (map<string, string>::iterator iter = orig_map.lower_bound(SEARCH_AMZ_PREFIX); iter != orig_map.end(); ++iter) {
const string& name = iter->first;
for (const auto& iter: orig_map) {

This comment has been minimized.

@theanalyst

theanalyst May 30, 2017

Member

hm this doesn't look right, use the lower bound, and maybe #define -> constexpr?

if (iter == env_map.end())
return def_val;
const char *s = iter->second.c_str();
return atoll(s);
return atoll(s);

This comment has been minimized.

@theanalyst

This comment has been minimized.

@theanalyst

theanalyst Jun 1, 2017

Member

since stol throws have to catch the exceptions as well

@theanalyst theanalyst changed the title from rgw: make RGWEnv return a const ref. to its map to wip: rgw: make RGWEnv return a const ref. to its map Jun 1, 2017

@theanalyst

This comment has been minimized.

Member

theanalyst commented Jun 1, 2017

will squash the PRs after reviews

#define SEARCH_AMZ_PREFIX "HTTP_X_AMZ_"
for (const auto& iter: orig_map) {
const string& name = iter.first;
#define SEARCH_AMZ_PREFIX "HTTP_X_AMZ_"

This comment has been minimized.

@rzarzynski

rzarzynski Jun 1, 2017

Contributor

What do you think about something like below?

static constexpr char SEARCH_AMZ_PREFIX[] = "HTTP_X_AMZ_";

This comment has been minimized.

@theanalyst
if (name == "HTTP_X_AMZ_DATE") /* dont forward date from original request */
continue;
if (name.compare(0, sizeof(SEARCH_AMZ_PREFIX) - 1, "HTTP_X_AMZ_") != 0)
if (name.compare(0, sizeof(SEARCH_AMZ_PREFIX) - 1, SEARCH_AMZ_PREFIX) != 0)

This comment has been minimized.

@rzarzynski

rzarzynski Jun 1, 2017

Contributor

I prefer to use std::strlen() instead of the sizeof() - 1. A reasonable compiler should optimise it out while it's a bit more descriptive and consistent with the case of having \0 in the middle of a char array. Though, this cosmetics is less than a suggestion.

This comment has been minimized.

@yehudasa

yehudasa Jun 1, 2017

Member

let's make sure that compiler actually optimizes it correctly before changing it

This comment has been minimized.

@rzarzynski

rzarzynski Jun 2, 2017

Contributor

The test program:

#include <cstring>
#include <iostream>

int main () {
  static constexpr char test_string[] = "Just a test C\0string";
  std::cout << "the length is: " << std::strlen(test_string) << std::endl;
}

Verification with GCC 4.8.4 on AMD64:

$ g++ -std=c++11 /tmp/test.cc -o /tmp/test
$ /tmp/test 
the length is: 13
$ objdump -dC /tmp/test

...

000000000040089d <main>:
  40089d:	55                   	push   %rbp
  40089e:	48 89 e5             	mov    %rsp,%rbp
  4008a1:	be c1 09 40 00       	mov    $0x4009c1,%esi
  4008a6:	bf 80 10 60 00       	mov    $0x601080,%edi
  4008ab:	e8 c0 fe ff ff       	callq  400770 <std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)@plt>
  4008b0:	be 0d 00 00 00       	mov    $0xd,%esi
  4008b5:	48 89 c7             	mov    %rax,%rdi
  4008b8:	e8 c3 fe ff ff       	callq  400780 <std::ostream::operator<<(unsigned long)@plt>
  4008bd:	be a0 07 40 00       	mov    $0x4007a0,%esi
  4008c2:	48 89 c7             	mov    %rax,%rdi
  4008c5:	e8 c6 fe ff ff       	callq  400790 <std::ostream::operator<<(std::ostream& (*)(std::ostream&))@plt>
  4008ca:	b8 00 00 00 00       	mov    $0x0,%eax
  4008cf:	5d                   	pop    %rbp
  4008d0:	c3                   	retq 

...

This comment has been minimized.

@yehudasa
@theanalyst

This comment has been minimized.

Member

theanalyst commented Jun 2, 2017

test this please

@theanalyst theanalyst changed the title from wip: rgw: make RGWEnv return a const ref. to its map to rgw: make RGWEnv return a const ref. to its map Jun 9, 2017

@theanalyst

This comment has been minimized.

Member

theanalyst commented Jun 9, 2017

@rzarzynski updated

@@ -454,7 +450,7 @@ int RGWRESTStreamWriteRequest::put_obj_init(RGWAccessKey& key, rgw_obj& obj, uin
new_info.script_uri.append(resource);
new_info.request_uri = new_info.script_uri;
map<string, string, ltstr_nocase>& m = new_env.get_map();

This comment has been minimized.

@rzarzynski

rzarzynski Jun 11, 2017

Contributor

Do we need this empty line?

This comment has been minimized.

@theanalyst

theanalyst Jun 12, 2017

Member

actually the next line also can be squashed if go for a range based for loop, pushing the changeset. I can always push back the previous version if we feel this is not necessary

for (iter = m.begin(); iter != m.end(); ++iter) {
headers.push_back(pair<string, string>(iter->first, iter->second));
const auto & m = new_env.get_map();
for (const auto& iter: m) {

This comment has been minimized.

@rzarzynski

rzarzynski Jun 11, 2017

Contributor

iter might be a bit confusing here. Maybe:

  for (const auto& kv: new_env.get_map()) {
    headers.emplace_back(kv);
  }

This comment has been minimized.

@theanalyst

theanalyst Jun 12, 2017

Member

thanks, more compact, did this in one other place, forgot here

}
map<string, string>& meta_map = new_info.x_meta_map;
for (iter = meta_map.begin(); iter != meta_map.end(); ++iter) {
headers.push_back(pair<string, string>(iter->first, iter->second));
for (const auto& iter: meta_map) {

This comment has been minimized.

@rzarzynski

rzarzynski Jun 11, 2017

Contributor

iter -> kv.

for (iter = meta_map.begin(); iter != meta_map.end(); ++iter) {
headers.push_back(pair<string, string>(iter->first, iter->second));
for (const auto& iter: meta_map) {
headers.push_back(pair<string, string>(iter.first, iter.second));

This comment has been minimized.

@rzarzynski

rzarzynski Jun 11, 2017

Contributor
    headers.emplace_back(kv);

This comment has been minimized.

@theanalyst

theanalyst Jun 12, 2017

Member

good catch

map<string, string>::iterator iter;
for (iter = m.begin(); iter != m.end(); ++iter) {
headers.push_back(pair<string, string>(iter->first, iter->second));
for (const auto& iter: new_env.get_map()) {

This comment has been minimized.

@rzarzynski

rzarzynski Jun 11, 2017

Contributor

iter -> kv.

map<string, string>::iterator iter;
for (iter = m.begin(); iter != m.end(); ++iter) {
headers.push_back(pair<string, string>(iter->first, iter->second));
for (const auto& iter: new_env.get_map()) {

This comment has been minimized.

@rzarzynski

rzarzynski Jun 11, 2017

Contributor

iter -> kv.

const size_t put_prefix_len = strlen(put_prefix);
const size_t del_prefix_len = strlen(del_prefix);
for (iter = m.begin(); iter != m.end(); ++iter) {
for (const auto& iter : s->info.env->get_map()) {

This comment has been minimized.

@rzarzynski

rzarzynski Jun 12, 2017

Contributor

iter -> kv.

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jun 12, 2017

@theanalyst: thanks for the update, Abhishek. I think this is the last iteration of review. The changes look good.

@theanalyst

This comment has been minimized.

Member

theanalyst commented Jun 12, 2017

@rzarzynski thanks for the patient reviews, updated, also changed a for loop in rgw_rest_client.cc L453 to a range based for loop while i was at it, let me know if this is okay, or I can always fall back to the previous version

for (iter = meta_map.begin(); iter != meta_map.end(); ++iter) {
headers.push_back(pair<string, string>(iter->first, iter->second));
for (const auto& kv: meta_map) {
headers.emplace_back(pair<string, string>(kv.first, kv.second));

This comment has been minimized.

@rzarzynski

rzarzynski Jun 12, 2017

Contributor

Maybe:

  for (const auto& kv: meta_map) {
    headers.emplace_back(kv);
  }

This comment has been minimized.

@theanalyst

theanalyst Jun 12, 2017

Member

sure, silly me

for (bliter = attrs.begin(); bliter != attrs.end(); ++bliter) {
bufferlist& bl = bliter->second;
const string& name = bliter->first;
for (auto attr: attrs) {

This comment has been minimized.

@rzarzynski

rzarzynski Jun 12, 2017

Contributor

auto&?

This comment has been minimized.

@theanalyst
@rzarzynski

LGTM. Thanks, @theanalyst! I'm scheduling a Teuthology run.

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jun 12, 2017

jenkins retest this please (Agent went offline during the build)

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jun 13, 2017

jenkins retest this please (cephtool-test-rados.sh)

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jun 14, 2017

@theanalyst: the Teuthology runs look clean:

Unfortunately, we've gotten a (trivial) merge conflict in rgw/rgw_crypt.cc. Could you please rebase?

rgw: make RGWEnv return a const ref. to its map
We already have a public method `set` for setting values in the RGW env
map, making the map as a read only. In addition:

- Added const to other  methods in `RGWEnv` which are getters
- `req_info::init_meta` also reused the same `iter` name for an internal
  find, changed the var name
- `add_grants_headers` now accepts an RGWEnv instead of the map
- use range based for loops wherever the code was changed
- req_info holds a ptr to a const RGWEnv, since we seem to only read the
  values from it

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
@theanalyst

This comment has been minimized.

Member

theanalyst commented Jun 14, 2017

@rzarzynski thanks! done

@rzarzynski

This comment has been minimized.

Contributor

rzarzynski commented Jun 14, 2017

jenkins retest this please (test_objectstore_memstore.sh)

@rzarzynski rzarzynski merged commit 04f9789 into ceph:master Jun 14, 2017

4 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
arm64 make check arm64 make check succeeded
Details
make check make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment