url-decode blowing up on round-trip url-encode/url-decode #73

Closed
jamieorc opened this Issue Jun 12, 2012 · 6 comments

Comments

Projects
None yet
2 participants
@jamieorc

I can url-encode an XML string. When I url-decode the returned value, I'm getting an index out-of-bounds on String in percent-decode (v 1.1.0):

xception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 3
at java.lang.String.charAt(String.java:686)
at java.util.regex.Matcher.appendReplacement(Matcher.java:711)
at clojure.string$replace_by.invoke(string.clj:58)
at clojure.string$replace.invoke(string.clj:82)
at ring.util.codec$percent_decode.doInvoke(codec.clj:31)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at ring.util.codec$url_decode.doInvoke(codec.clj:51)
at clojure.lang.RestFn.invoke(RestFn.java:410)

@jamieorc

This comment has been minimized.

Show comment Hide comment
@jamieorc

jamieorc Jun 12, 2012

Here's an example:

(c/url-decode (c/url-encode "<?xml version=\"1.0\" encoding=\"utf-8\"?>
<OTA_HotelAvailRQ RequestedCurrency=\"USD\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" Version=\"1.003\" xmlns=\"http://www.opentravel.org/OTA/2003/05\" xsi:schemaLocation=\"http://www.opentravel.org/2004B/OTA_HotelAvailRQ.xsd\" EchoToken=\"$xxxx\" PrimaryLangID=\"en\">
  <POS>
    <Source>
          <RequestorID Type=\"18\" URI=\"http://foobar.com\" ID=\"1234\"/>
    </Source>
  </POS>  
  <AvailRequestSegments>
    <AvailRequestSegment AvailReqType=\"Room\">
      <StayDateRange Start=\"1965-11-16\" End=\"1965-11-18\" />
      <RoomStayCandidates>
        <RoomStayCandidate Quantity=\"2\" NonSmoking=\"true\">
          <GuestCounts IsPerRoom=\"false\">
            <GuestCount AgeQualifyingCode=\"10\" Count=\"2\" />
          </GuestCounts>
        </RoomStayCandidate>
      </RoomStayCandidates>
      <HotelSearchCriteria><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id001\"/></Criterion><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id002\"/></Criterion><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id003\"/></Criterion></HotelSearchCriteria>
    </AvailRequestSegment>
  </AvailRequestSegments>
</OTA_HotelAvailRQ>"))

Here's an example:

(c/url-decode (c/url-encode "<?xml version=\"1.0\" encoding=\"utf-8\"?>
<OTA_HotelAvailRQ RequestedCurrency=\"USD\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" Version=\"1.003\" xmlns=\"http://www.opentravel.org/OTA/2003/05\" xsi:schemaLocation=\"http://www.opentravel.org/2004B/OTA_HotelAvailRQ.xsd\" EchoToken=\"$xxxx\" PrimaryLangID=\"en\">
  <POS>
    <Source>
          <RequestorID Type=\"18\" URI=\"http://foobar.com\" ID=\"1234\"/>
    </Source>
  </POS>  
  <AvailRequestSegments>
    <AvailRequestSegment AvailReqType=\"Room\">
      <StayDateRange Start=\"1965-11-16\" End=\"1965-11-18\" />
      <RoomStayCandidates>
        <RoomStayCandidate Quantity=\"2\" NonSmoking=\"true\">
          <GuestCounts IsPerRoom=\"false\">
            <GuestCount AgeQualifyingCode=\"10\" Count=\"2\" />
          </GuestCounts>
        </RoomStayCandidate>
      </RoomStayCandidates>
      <HotelSearchCriteria><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id001\"/></Criterion><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id002\"/></Criterion><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id003\"/></Criterion></HotelSearchCriteria>
    </AvailRequestSegment>
  </AvailRequestSegments>
</OTA_HotelAvailRQ>"))
@weavejester

This comment has been minimized.

Show comment Hide comment
@weavejester

weavejester Jun 12, 2012

Collaborator

Could you add this issue to https://github.com/ring-clojure/ring?

This repository is no longer the canonical Ring repository. Issues reported to this repo might be overlooked.

Collaborator

weavejester commented Jun 12, 2012

Could you add this issue to https://github.com/ring-clojure/ring?

This repository is no longer the canonical Ring repository. Issues reported to this repo might be overlooked.

@jamieorc

This comment has been minimized.

Show comment Hide comment
@jamieorc

jamieorc Jun 12, 2012

Yup, I realized it while searching for the problem. Also, we found the problem and I'll make note of it in ring-clojure/ring.

Yup, I realized it while searching for the problem. Also, we found the problem and I'll make note of it in ring-clojure/ring.

@jamieorc jamieorc closed this Jun 12, 2012

@jamieorc

This comment has been minimized.

Show comment Hide comment
@jamieorc

jamieorc Jun 12, 2012

Hey weavejester, issues are not yet turned on in ring-clojure/ring. BTW, the bug has to do with decoding of encoded $. It needs to be escaped.

Hey weavejester, issues are not yet turned on in ring-clojure/ring. BTW, the bug has to do with decoding of encoded $. It needs to be escaped.

@weavejester

This comment has been minimized.

Show comment Hide comment
@weavejester

weavejester Jun 12, 2012

Collaborator

Issues should now be enabled.

I've run across the "$" problem before. Last time it was because Java used special character sequences, $1, $2 etc., to refer to match groups in the value to be substituted. It screws up all kinds of things, and is fixed in the latest versions of Clojure. This just makes it harder to fix for older versions of Clojure, however, because we need to make sure any fix works for both old and new Clojure versions.

Collaborator

weavejester commented Jun 12, 2012

Issues should now be enabled.

I've run across the "$" problem before. Last time it was because Java used special character sequences, $1, $2 etc., to refer to match groups in the value to be substituted. It screws up all kinds of things, and is fixed in the latest versions of Clojure. This just makes it harder to fix for older versions of Clojure, however, because we need to make sure any fix works for both old and new Clojure versions.

@jamieorc

This comment has been minimized.

Show comment Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment