UrlCodingUtils.urlDecode() consumes all CPU

[keymon]

We are having trouble with our production systems that use scala+scalatra. Scalatra uses your library, rl.

We found in a situation where several threads end consuming 100% the CPU. The code is stuck in UrlCodingUtils.scala:71, that is basically the code from: https://github.com/backchatio/rl/blob/next/src/main/scala/rl/UrlCodingUtils.scala

I was reading the code from line 67, the while, and I got the impression that it might end in an endless loop:

while (in.hasRemaining) {   // While there is something there...
  val mark = in.position()    // Save the position...
  val c = in.get()
  if (c == '%') {                  // If there is a % ...
    if (in.remaining() >= 2) {
      val xc = in.get()
      val yc = in.get()
      val x = Character.digit(xc, 0x10)  // do  http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#digit%28char,%20int%29
      val y = Character.digit(yc, 0x10)
      if (x != -1 && y != -1) {               // If there is a problem
        val oo = (x << 4) + y
        if (!skip.contains(oo)) {
          out.put(oo.toByte)
        } else {
          out.put('%'.toByte)
          out.put(xc.toByte)
          out.put(yc.toByte)
        }
      } else {
        in.position(mark)    // RESET THE POSITION BACK where the problem was!!!
      }

My feeling is that we are getting a wrong url. From http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#digit%28char,%20int%29, it is a url with: %

If the radix is not in the range MIN_RADIX ≤ radix ≤ MAX_RADIX or if the value of ch is not a valid digit in the specified radix, -1 is returned. A character is a valid digit if at least one of the following is true:

    The method isDigit is true of the character and the Unicode decimal digit value of the character (or its single-character decomposition) is less than the specified radix. In this case the decimal digit value is returned.
    The character is one of the uppercase Latin letters 'A' through 'Z' and its code is less than radix + 'A' - 10. In this case, ch - 'A' + 10 is returned.
    The character is one of the lowercase Latin letters 'a' through 'z' and its code is less than radix + 'a' - 10. In this case, ch - 'a' + 10 is returned.
    The character is one of the fullwidth uppercase Latin letters A ('\uFF21') through Z ('\uFF3A') and its code is less than radix + '\uFF21' - 10. In this case, ch - '\uFF21' + 10 is returned.
    The character is one of the fullwidth lowercase Latin letters a ('\uFF41') through z ('\uFF5A') and its code is less than radix + '\uFF41' - 10. In this case, ch - '\uFF41' + 10 is returned. 

I was not able to reproduce this behaviour, because jetty always caches any invalid url I send, and now is late for more testing, but I think this is the origin of our problems.

The stack trace:

"qtp775489035-1002" prio=10 tid=0x00002aaabcfbc000 nid=0x10f9 runnable [0x0000000043844000]
   java.lang.Thread.State: RUNNABLE
        at rl.UrlCodingUtils$class.urlDecode(UrlCodingUtils.scala:71)
        at rl.UrlCodingUtils$.urlDecode(UrlCodingUtils.scala:110)
        at org.scalatra.UriDecoder$.firstStep(ScalatraBase.scala:19)
        at org.scalatra.ScalatraFilter$class.getRequestPath$1(ScalatraFilter.scala:51)
        at org.scalatra.ScalatraFilter$class.requestPath(ScalatraFilter.scala:58)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.requestPath(PasswordsController.scala:16)
        at org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:218)
        at org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:217)
        at scala.collection.immutable.Stream.flatMap(Stream.scala:442)
        at org.scalatra.ScalatraBase$class.runRoutes(ScalatraBase.scala:217)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.runRoutes(PasswordsController.scala:16)
        at org.scalatra.ScalatraBase$class.runActions$1(ScalatraBase.scala:163)
        at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply$mcV$sp(ScalatraBase.scala:175)
        at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:175)
        at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:175)
        at org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$cradleHalt(ScalatraBase.scala:190)
        at org.scalatra.ScalatraBase$class.executeRoutes(ScalatraBase.scala:175)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.executeRoutes(PasswordsController.scala:16)
        at org.scalatra.ScalatraBase$$anonfun$handle$1.apply$mcV$sp(ScalatraBase.scala:113)
        at org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:113)
        at org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:113)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
        at org.scalatra.DynamicScope$class.withResponse(DynamicScope.scala:80)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.withResponse(PasswordsController.scala:16)
        at org.scalatra.DynamicScope$$anonfun$withRequestResponse$1.apply(DynamicScope.scala:60)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
        at org.scalatra.DynamicScope$class.withRequest(DynamicScope.scala:71)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.withRequest(PasswordsController.scala:16)
        at org.scalatra.DynamicScope$class.withRequestResponse(DynamicScope.scala:59)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.withRequestResponse(PasswordsController.scala:16)
        at org.scalatra.ScalatraBase$class.handle(ScalatraBase.scala:111)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.org$scalatra$servlet$ServletBase$$super$handle(PasswordsController.scala:16)
        at org.scalatra.servlet.ServletBase$class.handle(ServletBase.scala:43)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.com$springer$core$app$CookieFlashMapHandler$$super$handle(PasswordsController.scala:16)
        at com.springer.core.app.CookieFlashMapHandler$class.handle(CookieFlashMapSupport.scala:24)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.org$scalatra$scalate$ScalateSupport$$super$handle(PasswordsController.scala:16)
        at org.scalatra.scalate.ScalateSupport$class.handle(ScalateSupport.scala:122)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.handle(PasswordsController.scala:16)
        at org.scalatra.ScalatraFilter$$anonfun$doFilter$1.apply$mcV$sp(ScalatraFilter.scala:33)
        at org.scalatra.ScalatraFilter$$anonfun$doFilter$1.apply(ScalatraFilter.scala:33)
        at org.scalatra.ScalatraFilter$$anonfun$doFilter$1.apply(ScalatraFilter.scala:33)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
        at org.scalatra.ScalatraFilter$class.doFilter(ScalatraFilter.scala:32)
        at com.springer.core.domaindriven.web.account.controllers.PasswordsController.doFilter(PasswordsController.scala:16)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)

[casualjim]

this issue is fixed in a later release

[casualjim]

fixed by these 2 commits:

scalatra@438c1e7
scalatra@0a5fd74