Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Whitespace in PUT bucket ACL XML payload causes errors #782

Closed
hectcastro opened this issue Jan 28, 2014 · 3 comments
Closed

Whitespace in PUT bucket ACL XML payload causes errors #782

hectcastro opened this issue Jan 28, 2014 · 3 comments
Labels
Bug
Milestone
1.4.5

Comments

@hectcastro
Copy link
Contributor

Below is some ngrep output that demonstrates a failure (with whitespace) and a success (without whitespace).

Also, if ngrep doesn't appear to be working on OS X 10.9, go here.

With whitespace and indentation:

T 127.0.0.1:59582 -> 127.0.0.1:8080 [AP]
PUT /mybucket/?acl HTTP/1.1.
User-Agent: fog/1.19.0.
Content-MD5: JLrUS1Nw01RLsz21YDEohA==.
Content-Type: application/json.
Date: Tue, 28 Jan 2014 18:44:49 +0000.
Authorization: AWS MOGMGYGVRHC7OYQHTLYS:6qtIOlQSP7qEPZW2sa7c67HgXsQ=.
Host: joker.dev:8080.
Content-Length: 529.
TE: trailers, deflate, gzip.
Connection: TE, close.
.


T 127.0.0.1:59582 -> 127.0.0.1:8080 [AP]
<AccessControlPolicy>
  <Owner>
    <ID>eb874c6afce06925157eda682f1b3c6eb0f3b983bbee3673ae62f41cce21f6b1</ID>
    <DisplayName>admin</DisplayName>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>eb874c6afce06925157eda682f1b3c6eb0f3b983bbee3673ae62f41cce21f6b1</ID>
        <DisplayName>admin</DisplayName>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

T 127.0.0.1:8080 -> 127.0.0.1:59582 [AP]
HTTP/1.1 500 Internal Server Error.
Server: Riak CS.
Date: Tue, 28 Jan 2014 18:44:49 GMT.
Content-Type: text/html.
Content-Length: 170.
.


T 127.0.0.1:8080 -> 127.0.0.1:59582 [AP]
<html><head><title>500 Internal Server Error</title></head><body><h1>Internal Server Error</h1>The server encountered an error while processing this request</body></html>

Without whitespace:

T 127.0.0.1:60336 -> 127.0.0.1:8080 [AP]
PUT /mybucket/?acl HTTP/1.1.
User-Agent: fog/1.19.0.
Content-MD5: r0Ahq+oAbExJxQj9DREqFg==.
Content-Type: application/json.
Date: Tue, 28 Jan 2014 19:42:04 +0000.
Authorization: AWS MOGMGYGVRHC7OYQHTLYS:VYeIIbMIqo+BCLEJIpKaiwCBNV4=.
Host: joker.dev:8080.
Content-Length: 457.
TE: trailers, deflate, gzip.
Connection: TE, close.
.


T 127.0.0.1:60336 -> 127.0.0.1:8080 [AP]
<AccessControlPolicy><Owner><ID>eb874c6afce06925157eda682f1b3c6eb0f3b983bbee3673ae62f41cce21f6b1</ID><DisplayName>admin</DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>eb874c6afce06925157eda682f1b3c6eb0f3b983bbee3673ae62f41cce21f6b1</ID><DisplayName>admin</DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>

T 127.0.0.1:8080 -> 127.0.0.1:60336 [AP]
HTTP/1.1 200 OK.
Server: Riak CS.
Date: Tue, 28 Jan 2014 19:42:04 GMT.
Content-Type: application/xml.
Content-Length: 0.
.

Props to the Cloud Foundry Blobstore Engineering team for helping us track this down:

image
@reiddraper
Copy link
Contributor

Note to myself: tried [{space, normalize}] as an option to xmerl_scan:string/2, but to no avail. The record being returned can't be parsed as an xmlElement, but it has the correct first atom, and is 12-elements long. Not sure what's going on there.

@kuenishi
Copy link
Contributor

kuenishi commented Feb 5, 2014

Looks like this is how xmerl works. We have to ignore all unexpected xmlText (and xmlComment) regardless of its size or else.

5> xmerl_scan:string("<html>      \n    </html>", [{space,normalize}]).
{{xmlElement,html,html,[],
             {xmlNamespace,[],[]},
             [],1,[],
             [{xmlText,[{html,1}],1,[]," ",text}],
             [],"/home/kuenishi/src/riak_cs",undeclared},
 []}
6> xmerl_scan:string("<html>      \n    </html>", []).                 
{{xmlElement,html,html,[],
             {xmlNamespace,[],[]},
             [],1,[],
             [{xmlText,[{html,1}],1,[],"      \n    ",text}],
             [],"/home/kuenishi/src/riak_cs",undeclared},
 []}

Doc says 'normalize' to accumulate consecutive whitespace and replace it with one space.

kuenishi added a commit that referenced this issue Feb 5, 2014
@kuenishi
add regression test for #782
8770ebc
@kuenishi kuenishi mentioned this issue Feb 5, 2014
Closed
kuenishi added a commit that referenced this issue Feb 6, 2014
@kuenishi
Handle well-indented XML including comments
569b657 
The root cause for #782 was misuse of xmerl_scan:string/2,
which doesn't emit spaces, comments between actual xml elements.
It leaves them as #xmlText{} and #xmlComment{} - This commit
changes to ignore them at walking around parsed dom tree. This
diffset also includes a small refactoring that wraps all
xmerl_scan:string/2 to one riak_cs_xml funcsion to control its
usage.

This also adds regression test as simple eunit test case.

Conflicts:
	src/riak_cs_acl_utils.erl
	src/riak_cs_s3_response.erl
@kuenishi kuenishi mentioned this issue Feb 6, 2014
Merged
@kuenishi kuenishi modified the milestones: 1.4.5, 1.5.0 Feb 6, 2014
@kuenishi
Copy link
Contributor

kuenishi commented Feb 8, 2014

Fixed via #795. Will be included in next version.

@kuenishi kuenishi closed this as completed Feb 8, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug
Projects
None yet
Milestone
1.4.5
Development

No branches or pull requests
3 participants
@kuenishi @hectcastro @reiddraper