Keys are doubly URL decoded, possibly key collision and list error

[shino]

Keys are doubly URL decoded at [1]

This possibly cause:

1. Key collision for originally different keys
2. List objects error even for ASCII only keys (together with Improvement to handle irregular strings as keyname. #910)
3. Keys in list objects are different from the ones at PUT Objects

Backward compatibility should be considerd in fixing this bug.

---

To illustrate, consider the case one uses a key %41
(3 bytes, the first letter is % (ASCII 0x25) at the appliction layer.

Use s3curl.pl here to look into TCP layer partially.
At HTTP layer, % should be URL encoded to %25, so it becomes like this.

% S3CURL=.s3curl.15018.alice s3curl.pl --id cs --put a.txt \
     -- -x 127.0.0.1:15018 -s \
     --trace-ascii - \
     'http://test.s3.amazonaws.com/%2541'
== Info: Hostname was NOT found in DNS cache
== Info:   Trying 127.0.0.1...
== Info: Connected to 127.0.0.1 (127.0.0.1) port 15018 (#0)
=> Send header, 302 bytes (0x12e)
0000: PUT http://test.s3.amazonaws.com/%2541 HTTP/1.1
0031: User-Agent: curl/7.35.0
004a: Host: test.s3.amazonaws.com
0067: Accept: */*
0074: Proxy-Connection: Keep-Alive
0092: x-amz-date: Thu, 25 Sep 2014 01:22:22 GMT
00bd: Authorization: AWS AS2NXQAZUZY2MEORZCBK:Z5yTIGMyc12DkKBmLg83ot5e
00fd: Vfg=
0103: Content-Length: 2
0116: Expect: 100-continue
012c:
<= Recv header, 23 bytes (0x17)
0000: HTTP/1.1 100 Continue
=> Send data, 2 bytes (0x2)
0000: a.
== Info: We are completely uploaded and fine
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
== Info: Server Riak CS is not blacklisted
<= Recv header, 17 bytes (0x11)
0000: Server: Riak CS
<= Recv header, 42 bytes (0x2a)
0000: ETag: "60b725f10c9c85c70d97880dfe8191b3"
<= Recv header, 37 bytes (0x25)
0000: Date: Thu, 25 Sep 2014 01:22:22 GMT
<= Recv header, 26 bytes (0x1a)
0000: Content-Type: text/plain
<= Recv header, 19 bytes (0x13)
0000: Content-Length: 0
<= Recv header, 2 bytes (0x2)
0000:
== Info: Connection #0 to host 127.0.0.1 left intact

When listing objects , key turns into A (1 byte):

% S3CURL=.s3curl.15018.alice s3curl.pl --id cs -- -x 127.0.0.1:15018 -s --trace-ascii - 'http://test.s3.amazonaws.com/'
== Info: Hostname was NOT found in DNS cache
== Info:   Trying 127.0.0.1...
== Info: Connected to 127.0.0.1 (127.0.0.1) port 15018 (#0)
=> Send header, 256 bytes (0x100)
0000: GET http://test.s3.amazonaws.com/ HTTP/1.1
002c: User-Agent: curl/7.35.0
0045: Host: test.s3.amazonaws.com
0062: Accept: */*
006f: Proxy-Connection: Keep-Alive
008d: x-amz-date: Thu, 25 Sep 2014 01:25:38 GMT
00b8: Authorization: AWS AS2NXQAZUZY2MEORZCBK:stDmA/wO9mcVa7cJsKiWtD42
00f8: 0gw=
00fe:
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
== Info: Server Riak CS is not blacklisted
<= Recv header, 17 bytes (0x11)
0000: Server: Riak CS
<= Recv header, 37 bytes (0x25)
0000: Date: Thu, 25 Sep 2014 01:25:38 GMT
<= Recv header, 31 bytes (0x1f)
0000: Content-Type: application/xml
<= Recv header, 21 bytes (0x15)
0000: Content-Length: 556
<= Recv header, 2 bytes (0x2)
0000:
<= Recv data, 556 bytes (0x22c)
0000: <?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="h
0040: ttp://s3.amazonaws.com/doc/2006-03-01/"><Name>test</Name><Prefix
0080: ></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><Delimiter></D
00c0: elimiter><IsTruncated>false</IsTruncated><Contents><Key>A</Key><   ##<=== HERE
0100: LastModified>2014-09-25T01:22:22.000Z</LastModified><ETag>"60b72
0140: 5f10c9c85c70d97880dfe8191b3"</ETag><Size>2</Size><StorageClass>S
0180: TANDARD</StorageClass><Owner><ID>079ffb96b6877cfe269f2efe3c0e876
01c0: 4478f6482cbd448905f517d4c384e8983</ID><DisplayName>alice</Displa
0200: yName></Owner></Contents></ListBucketResult>
<?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>test</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><Delimiter></Delimiter><IsTruncated>false</IsTruncated><Contents><Key>A</Key><LastModified>2014-09-25T01:22:22.000Z</LastModified><ETag>"60b725f10c9c85c70d97880dfe8191b3"</ETag><Size>2</Size><StorageClass>STANDARD</StorageClass><Owner><ID>079ffb96b6877cfe269f2efe3c0e8764478f6482cbd448905f517d4c384e8983</ID><DisplayName>alice</DisplayName></Owner></Contents></ListBucketResult>== Info: Connection #0 to host 127.0.0.1 left intact

If one put the key %26FF (at application layer) which contains ASCII only,
list objects fails for the same reason as #910.

[1]

riak_cs/src/riak_cs_wm_utils.erl

Line 408 in 6a7728a

Key = mochiweb_util:unquote(mochiweb_util:unquote(wrq:path_info(object, RD))),

[shino]

Addressed by #1040