Skip to content

thread safety of reflection API #6240

Closed

Description

@scabug

It's sort of implemented, but was never tested. Also trees aren't synchronized at all, because it was only recently when they've been converted to abstract types.

NOTE that this issue mostly affects runtime reflection, not reflection used in macros. Compiler itself has never been thread-safe, so noone uses it from multiple threads, therefore issues might only arise if a macro spawns multiple threads, which do certain operations simultaneously.

Activity

  1. scabug commented on Aug 15, 2012

    @scabug
    Author

    Imported From: https://issues.scala-lang.org/browse/SI-6240?orig=1
    Reporter: @xeno-by
    Affected Versions: 2.10.0

  2. scabug commented on Sep 6, 2012

    @scabug
    Author

    @xeno-by said:
    Also synchronize WellKnownSymbols

  3. scabug commented on Sep 26, 2012

    @scabug
    Author

    @xeno-by said:
    Most symbols aren't synchronized right now, because synchronization is implemented by decoration, and decorators aren't applied to almost anything: scala/scala#1380

  4. scabug commented on Nov 15, 2012

    @scabug
    Author

    Kartik Subramanian (selfification) said:
    How thread safe is reflection as it stands right now? If it is not really thread-safe, is there a way to globally synchronize all calls into the API?

    I ask this because I have code that tries to use reflection to fetch modules that are a subtype of some given type. During testing, if I run my tests in parallel (using sbt test), I get various failures like NPEs, "scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving value", unsupported operation exception for "tail of empty list", all of them coming from reflect.internal.SymbolTable or from reflect.internal.Symbols$Symbol.lock. It's highly non-reproducible and I don't know enough about reflection to reduce it to a minimal test case. If I run the tests linearly (using sbt test-only name-of-test), each of them individually pass. If you guys are already aware that Symbol is not thread-safe, I don't want to spend time trying to simplify my issue to a bug you are already aware of.

    Just as an aside, even when running the tests serially, I get printouts like:
    [] ?$2 setInst scala.List[scala.Int]
    [] ?
    $2 setInst scala.List[scala.Int]
    and
    [] !!! MyClass[String].transform(MyClass MyClass), but tparams.isEmpty and args=
    [] !!! MyClass[T].transform(MyClass MyClass), but tparams.isEmpty and args=
    I don't know if this is relevant to the thread-safety issue. I am trying to reflectively acquire objects with types that look like MyClass[_] - that may be triggering some code path that isn't thread-safe yet?

    Let me know if you want more info or if you want to see the source.

  5. scabug commented on Dec 6, 2012

    @scabug
    Author

    @xeno-by said:
    Another case study by hejfelix (https://github.com/scala/scala/issues/1723):

    I am using scala in the eclipse plugin version 2.1.0.m2 and I am getting problems when using the reflection API in a concurrent setting.

    I have a big framework using MPJ-Express as the backend and running applications in multi-core mode. When I use the reflection api, I get random occurences of the runtime eror displayed below:

    java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at runtime.starter.MulticoreStarter$1.run(MulticoreStarter.java:289)
    at java.lang.Thread.run(Thread.java:722)
    Caused by: scala.reflect.runtime.ReflectError: value Predef is not a package
    at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$$makeScalaPackage(JavaMirrors.scala:893)
    at scala.reflect.runtime.JavaMirrors$class.missingHook(JavaMirrors.scala:1256)
    at scala.reflect.runtime.JavaUniverse.missingHook(JavaUniverse.scala:12)
    at scala.reflect.internal.Mirrors$RootsBase.universeMissingHook(Mirrors.scala:77)
    at scala.reflect.internal.Mirrors$RootsBase.missingHook(Mirrors.scala:79)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
    at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass(Mirrors.scala:72)
    at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:161)
    at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:21)
    at it.vigtig.thesis.test.NGridTest$$typecreator2$1.apply(NGridTest.scala:103)
    at scala.reflect.api.TypeTags$TypeTag$.apply(TypeTags.scala:302)
    at it.vigtig.thesis.test.NGridTest$.testGrid(NGridTest.scala:103)
    at it.vigtig.thesis.test.NGridTest$$anonfun$main$1.apply$mcV$sp(NGridTest.scala:16)
    at it.vigtig.thesis.env.package$$anonfun$parallelize$1.apply$mcV$sp(package.scala:68)
    at it.vigtig.thesis.env.comm.mpj.MPJDistributorTrait$class.execute(MPJDistributorTrait.scala:17)
    at it.vigtig.thesis.env.comm.mpj.MPJDistributor$.execute(MPJDistributor.scala:3)
    at it.vigtig.thesis.env.package$.parallelize(package.scala:59)
    at it.vigtig.thesis.test.NGridTest$.main(NGridTest.scala:14)
    at it.vigtig.thesis.test.NGridTest.main(NGridTest.scala)
    ... 6 more

    I am not sharing any reflection knowledge or objects between threads, so this behaviour must be a bug. The parts I am using from reflection are TypeTags and ClassTags as implicit arguments.

  6. scabug commented on Dec 6, 2012

    @scabug
    Author

    @paulp said:
    "value Predef is not a package" suggests that it isn't even blocking for Definitions.init() to complete, is that true? If that's the case then people had better put their concurrent reflection aspirations on ice until at least 2.10.1.

  7. scabug commented on Dec 6, 2012

    @scabug
    Author

    @xeno-by said:
    Looks like it's not blocking indeed.

  8. scabug commented on Dec 25, 2012

    @scabug
    Author

    @xeno-by said (edited on Dec 25, 2012 10:29:23 PM UTC):
    To quote Paul Butcher,

    Actually - I take that back. I have one test of this form for the old non-type-macro-based version of mocks and one for the typemacro-based version. And they're in different suites, so they might be running parallel with each other.

    But if this is the problem, then I suspect we really need to reconsider the thread safety of reflection. If just adding a TypeTag context bound to a method is enough to make it non-threadsafe, then it's bound to catch people out.

    Raising the priority.

    upd. Discussion: http://groups.google.com/group/scala-internals/browse_thread/thread/58d62991c21ba2f8.

    > [info] - cope with context bounds *** FAILED ***
    > [info]   java.lang.NullPointerException:
    > [info]   at scala.reflect.internal.Types$TypeRef.computeHashCode(Types.scala:2332)
    > [info]   at scala.reflect.internal.Types$UniqueType.<init>(Types.scala:1274)
    > [info]   at scala.reflect.internal.Types$TypeRef.<init>(Types.scala:2315)
    > [info]   at scala.reflect.internal.Types$NoArgsTypeRef.<init>(Types.scala:2107)
    > [info]   at scala.reflect.internal.Types$ModuleTypeRef.<init>(Types.scala:2078)
    > [info]   at scala.reflect.internal.Types$PackageTypeRef.<init>(Types.scala:2095)
    > [info]   at scala.reflect.internal.Types$TypeRef$.apply(Types.scala:2516)
    > [info]   at scala.reflect.internal.Types$class.typeRef(Types.scala:3577)
    > [info]   at scala.reflect.internal.SymbolTable.typeRef(SymbolTable.scala:13)
    > [info]   at scala.reflect.internal.Symbols$TypeSymbol.newTypeRef(Symbols.scala:2754)
  9. scabug commented on Jan 2, 2013

    @scabug
    Author

    @OlegYch said:
    isn't there any way to init that upfront before normal program run?

  10. scabug commented on Jan 2, 2013

    @scabug
    Author

    @xeno-by said:
    There is. Just calling scala.reflect.runtime.universe should be enough.

  11. scabug commented on Jan 2, 2013

    @scabug
    Author

    Felix Palludan Hargreaves (hejfelix) said:
    Doesn't work for me. I get the following:

    java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at runtime.starter.MulticoreStarter$1.run(MulticoreStarter.java:289)
    at java.lang.Thread.run(Thread.java:722)
    Caused by: java.lang.UnsupportedOperationException: tail of empty list
    at scala.collection.immutable.Nil$.tail(List.scala:330)
    at scala.collection.immutable.Nil$.tail(List.scala:325)
    at scala.reflect.internal.SymbolTable.popPhase(SymbolTable.scala:166)
    at scala.reflect.internal.pickling.UnPickler$Scan$LazyTypeRef.complete(UnPickler.scala:856)
    at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1217)
    at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:238)
    at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:294)
    at scala.reflect.runtime.SymbolLoaders$LazyPackageType.complete(SymbolLoaders.scala:88)
    at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1217)
    at scala.reflect.internal.Types$TypeRef.thisInfo(Types.scala:2364)
    at scala.reflect.internal.Types$TypeRef.baseClasses(Types.scala:2369)
    at scala.reflect.internal.Types$Type.findMember(Types.scala:1138)
    at scala.reflect.internal.Types$Type.memberBasedOnName(Types.scala:687)
    at scala.reflect.internal.Types$Type.member(Types.scala:645)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:43)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
    at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass(Mirrors.scala:72)
    at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:119)
    at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:21)
    at it.vigtig.thesis.test.MatrixBench$$typecreator2$1.apply(MatrixBench.scala:149)
    at scala.reflect.api.TypeTags$TypeTag$.apply(TypeTags.scala:302)
    at it.vigtig.thesis.test.MatrixBench$.mul(MatrixBench.scala:149)
    at it.vigtig.thesis.test.MatrixBench$.bench(MatrixBench.scala:139)
    at it.vigtig.thesis.test.MatrixBench$$anonfun$main$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$1.apply$mcVI$sp(MatrixBench.scala:94)
    at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:142)
    at it.vigtig.thesis.test.MatrixBench$$anonfun$main$1$$anonfun$apply$mcV$sp$1.apply(MatrixBench.scala:93)
    at it.vigtig.thesis.test.MatrixBench$$anonfun$main$1$$anonfun$apply$mcV$sp$1.apply(MatrixBench.scala:90)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1156)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at it.vigtig.thesis.test.MatrixBench$$anonfun$main$1.apply$mcV$sp(MatrixBench.scala:90)
    at it.vigtig.thesis.env.package$$anonfun$parallelize$1.apply$mcV$sp(package.scala:72)
    at it.vigtig.thesis.env.comm.mpj.MPJDistributorTrait$class.execute(MPJDistributorTrait.scala:17)
    at it.vigtig.thesis.env.comm.mpj.MPJDistributor$.execute(MPJDistributor.scala:3)
    at it.vigtig.thesis.env.package$.parallelize(package.scala:63)
    at it.vigtig.thesis.test.MatrixBench$.main(MatrixBench.scala:35)
    at it.vigtig.thesis.test.MatrixBench.main(MatrixBench.scala)
    ... 6 more

  12. scabug commented on Jan 2, 2013

    @scabug
    Author

    @xeno-by said:
    Could you please tell more about your scenario? Probably the best way to communicate would be email - just google me, and there'll also be my email.

  13. scabug commented on Jan 3, 2013

    @scabug
    Author

    @eengbrec said:
    If you checkout this branch of this relatively small project, comment out "parallelExecution in Test := false " in build.sbt, then run the tests you should see non-deterministic failures. Just run test repeatedly and watch.

    https://github.com/eengbrec/optional/tree/scala210-apachecli

  14. scabug commented on Jan 3, 2013

    @scabug
    Author

    @xeno-by said:
    Thanks! This will be one of the first things I'll get to after my vacation

  15. scabug commented on Jan 8, 2013

    @scabug
    Author
  16. 40 remaining items

  17. scabug commented on Feb 2, 2015

    @scabug
    Author

    @retronym said:
    Erasure isn't to blame here, in fact it is rather crucial for a language like Scala to be able to innovate in its type system. Compare this to F#, for example, which hasn't been able to add type constructor polymorphism due to the assumptions about generics built into the VM.

  18. added this to the 2.11.0-M7 milestone on Apr 7, 2017
  19. SethTisue commented on Jun 13, 2020

    @SethTisue
    Member

    https://docs.scala-lang.org/overviews/reflection/thread-safety.html says "Thread safety issues have been fixed in Scala 2.11.0-RC1", with a link to this ticket.

    @nogurenn asked on Gitter so I did a little research about where things stand today.

    the most negative thing I found is #10766, particularly @adriaanm's remark at #10766 (comment) :

    scala.reflect is experimental. Making it thread safe without killing performance is nearly impossible. I was against even trying. The workaround is to synchronize access by the user, or not use it at all

    and then at #10766 (comment) :

    Just to be clear, we do not intend to provide (full) thread-safety for scala.reflect -- I don't think it's feasible

    on the other hand, improvements to thread safety do sometimes still land, e.g. @retronym's recent scala/scala#8433 . (I believe this was work for a customer.) and we're certainly open to merging contributed fixes in this area

    still-open bugs in this area include #10598 and #10431; perhaps there are others

  20. SethTisue commented on Jun 13, 2020

    @SethTisue
    Member

    SethTisue unassigned xeno-by 3 minutes ago

    uh, weird, I didn't intend to, and I don't understand why I can't undo the change, either :-/

  21. deleted a comment from scabug on Feb 9, 2023
  22. added 5 commits that reference this issue on Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions