New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ScalaClassLoader.asContext highjacks the thread context classloader and causes classloader branching that destroy proper parent relationship #9587

Closed
scabug opened this Issue Dec 11, 2015 · 1 comment

Comments

Projects
None yet
2 participants
@scabug
Copy link

scabug commented Dec 11, 2015

ScalaClassLoader.asContext highjacks the thread context classloader and causes classloader branching that destroy proper parent relationship

The thing I tried solving was running some runtime compilation that would allow some parts of my configuration be written in scala code (essentially the config is a function that maps stuff).

To get that working this is what I did.

val factory = new IMain.Factory()
val engine = factory.getScriptEngine().asInstanceOf[IMain]
val cl = java.lang.Thread.currentThread.getContextClassLoader
engine.settings.embeddedDefaults(cl)
engine.settings.usejavacp.value = true

val sf = new BatchSourceFile(new PlainFile(Path(s"local.conf/${script}")))
engine.compileSources(sf)

val fun = engine.eval(customConf.getString(GENERATOR_FUNCTION))
   .asInstanceOf[(java.util.Map[String, Object] => java.util.Map[String, Object])]

This code worked in an isolated test program, but once I started combining the thing with real code everything broke down. Example issue:

Caused by: org.apache.velocity.exception.VelocityException: Failed to initialize an instance of org.apache.velocity.runtime.log.NullLogChute with the current runtime configuration.
    at org.apache.velocity.runtime.log.LogManager.createLogChute(LogManager.java:220)
    at org.apache.velocity.runtime.log.LogManager.updateLog(LogManager.java:269)
    at org.apache.velocity.runtime.RuntimeInstance.initializeLog(RuntimeInstance.java:871)
    ... 18 more
Caused by: org.apache.velocity.exception.VelocityException: The specified logger class org.apache.velocity.runtime.log.NullLogChute does not implement the org.apache.velocity.runtime.log.LogChute interface.
    at org.apache.velocity.runtime.log.LogManager.createLogChute(LogManager.java:181)
    ... 20 more

Those two classes come from the same jar, so typing between them should not ever be a problem. To me it looked like a Classloader issue, and it was.

Turns out the org.apache.velocity.runtime.log.LogChute interface was loaded by:
sun.misc.Launcher$AppClassLoader (1 instance in heap)
org.apache.velocity.runtime.log.NullLogChute and second copy of LogChute where loaded by:
scala.reflect.internal.util.ScalaClassLoader$URLClassLoader (1 instance in heap)
Both classloaders had a common parent:
sun.misc.Launcher$ExtClassLoader
which makes them classloader siblings explaining the incompatible types.

I thought it was going wrong in the compile part, but as it turns out, it breaks in the eval() part.
At some point this sequence of things is called (not full stack listed, top down order).
scala.tools.nsc.interpreter.IMain.WrappedRequest.loadAndRunReq
scala.reflect.internal.util.ScalaClassLoader.asContext
scala.reflect.internal.util.ScalaClassLoader.apply(cl: ClassLoader)

These are the instresting bits
scala.reflect.internal.util.ScalaClassLoader.asContext

  def asContext[T](action: => T): T = {
    val saved = contextLoader
    try { setContext(this) ; action }
    finally setContext(saved)
  }

scala.reflect.internal.util.ScalaClassLoader.contextLoader

def contextLoader = apply(Thread.currentThread.getContextClassLoader)

scala.reflect.internal.util.ScalaClassLoader.apply(cl: ClassLoader)

  implicit def apply(cl: JClassLoader): ScalaClassLoader = cl match {
    case cl: ScalaClassLoader => cl
    case cl: JURLClassLoader  => new URLClassLoader(cl.getURLs.toSeq, cl.getParent)
    case _                    => new JClassLoader(cl) with ScalaClassLoader
  }

Now what happens here is:

  • When the context class loader happens to be a java URLClassloader, it gets stripped and repackaged in a scala URLClassloader, with reuse of the parent of the original loader
  • This destroys a meaningful parent relationship that should have been there, allowing commons stuff to be loaded by a common Classloader
  • It also replaces what is 'saved' in the asContext utility
  • Hence it is what is restored as thread context class loader once asContext is done

Which led to the breakage described above.

With this insight I was able to build this workarround (it prepackages the context class loader as the fallback case does in the above code):

  val t = java.lang.Thread.currentThread
  val cl = t.getContextClassLoader
  try {
    val clplus = new java.lang.ClassLoader(cl) with ScalaClassLoader
    t.setContextClassLoader(clplus)

    val factory = new IMain.Factory()
    val engine = factory.getScriptEngine().asInstanceOf[IMain]
    engine.settings.embeddedDefaults(clplus)
    engine.settings.usejavacp.value = true

    val sf = new BatchSourceFile(new PlainFile(Path(s"local.conf/${script}")))
    engine.compileSources(sf)

    val fun = engine.eval(customConf.getString(GENERATOR_FUNCTION))
      .asInstanceOf[(java.util.Map[String, Object] => java.util.Map[String, Object])]

    engine.close()

    Some(fun) // returning the function here
  } finally {
    t.setContextClassLoader(cl)
  }

Though I'm glad I could work around it, I can't help at feel this smells like a bug.

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Dec 11, 2015

Imported From: https://issues.scala-lang.org/browse/SI-9587?orig=1
Reporter: Jean-Luc Deprez (spangaer)
Affected Versions: 2.11.7
Duplicates #8521

@scabug scabug added repl build labels Apr 7, 2017

@som-snytt som-snytt closed this Dec 12, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment