Fails when secure=False: FoldCaseDict instance has no attribute 'copy'

[hashbackup]

Here's a traceback:

  File "minio/api.py", line 958, in list_objects
    headers=headers)
  File "minio/api.py", line 1845, in _url_open
    preload_content=preload_content)
  File "urllib3/poolmanager.py", line 161, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "urllib3/connectionpool.py", line 538, in urlopen
    headers = headers.copy()
AttributeError: FoldCaseDict instance has no attribute 'copy'

I'm using urllib3 that comes bundled with requests version 2.8.1. I wasn't sure the right way to handle this since minio also wants to import urllib3 directly, so made a symlink: ln -s requests/packages/urllib3 . This is for a static build of HashBackup.

[harshavardhana]

File "minio/api.py", line 958, in list_objects
headers=headers)
File "minio/api.py", line 1845, in _url_open
preload_content=preload_content)
File "urllib3/poolmanager.py", line 161, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "urllib3/connectionpool.py", line 538, in urlopen
headers = headers.copy()
AttributeError: FoldCaseDict instance has no attribute 'copy'

Strange this should be working.. secure or otherwise shouldn't have any problems.

[hashbackup]

I downloaded the latest release of urllib3. Using that, here is the traceback:

Traceback (most recent call last):
  File "mintest.py", line 27, in <module>
    for obj in objects:
  File "minio/api.py", line 958, in list_objects
    headers=headers)
  File "minio/api.py", line 1845, in _url_open
    preload_content=preload_content)
  File "urllib3/poolmanager.py", line 244, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "urllib3/connectionpool.py", line 570, in urlopen
    headers = headers.copy()
AttributeError: FoldCaseDict instance has no attribute 'copy'

I thought maybe they removed this copy(), but guess they didn't. Works fine with secure=True, so I guess they don't do a copy for that path. The new case-independent dict apparently doesn't support the copy() method like a regular dict:

>>> from minio.fold_case_dict import FoldCaseDict
>>> d=FoldCaseDict()
>>> x=d.copy()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: FoldCaseDict instance has no attribute 'copy'
>>> d={}
>>> x=d.copy()
>>> 

There was a discussion on StackOverflow about case-insensitive dicts: http://stackoverflow.com/questions/2082152/case-insensitive-dictionary

(Edited to remove pathnames)

[harshavardhana]

Ah found the problem, our mistake.

-class FoldCaseDict():
+class FoldCaseDict(dict):
     def __init__(self, dictionary={}):
         self._data = self.__create(dictionary)

[harshavardhana]

Can you try this? @hashbackup - #489

[hashbackup]

Hmm... now get this:

Traceback (most recent call last):
  File "mintest.py", line 27, in <module>
    for obj in objects:
  File "minio/api.py", line 958, in list_objects
    headers=headers)
  File "minio/api.py", line 1868, in _url_open
    raise response_error.get(bucket_name, object_name)
minio.error.ResponseError: ResponseError: code: AccessDenied, message: Access Denied, bucket_name: xxx, object_name: None, request_id: xxx, host_id: xxx, region: us-east-1

I restored the previous release of minio and it worked fine. Not sure what's wrong. I did a trace, but it only shows a 403 error, which isn't too helpful.

[hashbackup]

secure=True still works fine.

I took a look at urllib3, at that copy() that was failing before. Printed headers before the copy, and headers after the copy. On the requests that worked, they were the equivalent (order changed, no big deal) and they had that "title case".

On the request that failed, the headers before the copy had keys that were all lowercase, and copying that returned a null dict.

[hashbackup]

Here ya go:

>>> from minio.fold_case_dict import FoldCaseDict
>>> d = FoldCaseDict()
>>> d['a'] = 'xyz'
>>> d
{'a': 'xyz'}
>>> d = d.copy()
>>> d
{}

Not such a good copy...

[hashbackup]

One problem I think is that when a FoldCaseDict is copied, it's returning a regular dict.

>>> type(d)
<type 'dict'>

[harshavardhana]

No i need to implement a function specific for this.. Let me do that, wanted to see if it can be avoided looks like it is not possible.