url-decode blowing up on round-trip url-encode/url-decode

[jamieorc]

I can url-encode an XML string. When I url-decode the returned value, I'm getting an index out-of-bounds on String in percent-decode (v 1.1.0):

xception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 3
at java.lang.String.charAt(String.java:686)
at java.util.regex.Matcher.appendReplacement(Matcher.java:711)
at clojure.string$replace_by.invoke(string.clj:58)
at clojure.string$replace.invoke(string.clj:82)
at ring.util.codec$percent_decode.doInvoke(codec.clj:31)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at ring.util.codec$url_decode.doInvoke(codec.clj:51)
at clojure.lang.RestFn.invoke(RestFn.java:410)

[jamieorc]

Here's an example:

(c/url-decode (c/url-encode "<?xml version=\"1.0\" encoding=\"utf-8\"?>
<OTA_HotelAvailRQ RequestedCurrency=\"USD\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" Version=\"1.003\" xmlns=\"http://www.opentravel.org/OTA/2003/05\" xsi:schemaLocation=\"http://www.opentravel.org/2004B/OTA_HotelAvailRQ.xsd\" EchoToken=\"$xxxx\" PrimaryLangID=\"en\">
  <POS>
    <Source>
          <RequestorID Type=\"18\" URI=\"http://foobar.com\" ID=\"1234\"/>
    </Source>
  </POS>  
  <AvailRequestSegments>
    <AvailRequestSegment AvailReqType=\"Room\">
      <StayDateRange Start=\"1965-11-16\" End=\"1965-11-18\" />
      <RoomStayCandidates>
        <RoomStayCandidate Quantity=\"2\" NonSmoking=\"true\">
          <GuestCounts IsPerRoom=\"false\">
            <GuestCount AgeQualifyingCode=\"10\" Count=\"2\" />
          </GuestCounts>
        </RoomStayCandidate>
      </RoomStayCandidates>
      <HotelSearchCriteria><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id001\"/></Criterion><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id002\"/></Criterion><Criterion><HotelRef ChainCode=\"BBCom\" HotelCode=\"id003\"/></Criterion></HotelSearchCriteria>
    </AvailRequestSegment>
  </AvailRequestSegments>
</OTA_HotelAvailRQ>"))

[weavejester]

Could you add this issue to https://github.com/ring-clojure/ring?

This repository is no longer the canonical Ring repository. Issues reported to this repo might be overlooked.

[jamieorc]

Yup, I realized it while searching for the problem. Also, we found the problem and I'll make note of it in ring-clojure/ring.

[jamieorc]

Hey weavejester, issues are not yet turned on in ring-clojure/ring. BTW, the bug has to do with decoding of encoded $. It needs to be escaped.

[weavejester]

Issues should now be enabled.

I've run across the "$" problem before. Last time it was because Java used special character sequences, $1, $2 etc., to refer to match groups in the value to be substituted. It screws up all kinds of things, and is fixed in the latest versions of Clojure. This just makes it harder to fix for older versions of Clojure, however, because we need to make sure any fix works for both old and new Clojure versions.

[jamieorc]

See the notes here: http://clojuredocs.org/clojure_core/clojure.string/replace