Getting a private field using reflection fails when name-mangled for access

[scabug]

Test case below. Uncommenting the line makes the test throw:

scala.ScalaReflectionException: Scala field x  isn't represented as a Java field, neither it has a Java accessor method
note that private parameters of class constructors don't get mapped onto fields and/or accessors,
unless they are used outside of their declaring constructors.

Test case:

class FieldAccessTest {

  class TestClass {
    private val x = 123
    // Uncomment the following line to make the test fail
    // () => x
  }

  @Test
  def testFieldAccess(): Unit = {
    import scala.reflect.runtime.{universe => ru}
    val mirror = ru.runtimeMirror(getClass.getClassLoader)

    val obj = new TestClass
    val objType = mirror.reflect(obj).symbol.toType
    val objFields = objType.members.collect { case ms: ru.MethodSymbol if ms.isGetter => ms }

    Assert.assertEquals(123, mirror.reflect(obj).reflectField(objFields.head).get)
  }
}

The problem appears to be caused by name mangling, which is triggered by the presence of the anonymous function. The field is then named "FieldAccessTest$TestClass$$x" instead of "x" in the resulting bytecode, which causes reflection to fail.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-9306?orig=1
Reporter: Marko Kolar (mkolar)
Affected Versions: 2.11.6, 2.11.7

[scabug]

Mike (mike) said:
I'm testing with a 2.11.8 snapshot (7a950df664405ff9abe5068f70f6e2e6d586c541), and OpenJDK 1.8.0_66-internal-b01.

Without the () => x, javap gives:

public class scala.reflect.FieldAccessTest$TestClass {
  private final int x;
  ...
  private int x();
  ...
}

With it:

  private final int scala$reflect$FieldAccessTest$TestClass$$x;
  ...
  public int scala$reflect$FieldAccessTest$TestClass$$x();

JavaMirrors.fieldToJava() is called from reflectField(), and it calls jclazz getDeclaredField expandedName(fld) in line 1261:

def fieldToJava(fld: TermSymbol): jField = fieldCache.toJava(fld) {
  val jclazz = classToJava(fld.owner.asClass)
  val jname = fld.name.dropLocal.toString
  try jclazz getDeclaredField jname
  catch {
    case ex: NoSuchFieldException => jclazz getDeclaredField expandedName(fld)
  }
}

Inside Class.getDeclaredField(), Class.searchFields() is called and fails to match the expanded name scala$reflect$FieldAccessTest$TestClass$$x to the interned name on line 2958:

private static Field searchFields(Field[] fields, String name) {
    String internedName = name.intern();
    for (int i = 0; i < fields.length; i++) {
        if (fields[i].getName() == internedName) {
            return getReflectionFactory().copyField(fields[i]);
        }
    }
    return null;
}

Which leads me to believe that the expanded names are not being interned.

[scabug]

Mike (mike) said:
The error these days is:

scala.ScalaReflectionException: Scala field x  of class TestClass isn't represented as a Java field, nor does it have a
Java accessor method. One common reason for this is that it may be a private class parameter
not used outside the primary constructor.

[scabug]

Alexey Kudinkin (alexeykudinkin) said:
Same issue here impeding scala.pickling to serialise object due transient private field.

trait Logging {
  @transient private var log_ : Logger = null

  // ...
}

[scabug]

Tanju Cataltepe (tanjucat) said:
I'm getting the same error while unpickling a case class with some private val s. I'm getting the error for only one of the fields even though there are other private and non-private val s of the same type ( Map[Int, Double] ). I tried changing the order of declaration of the fields in the class, changed the order of pickling the fields, still the same field causes the failure:

scala.ScalaReflectionException: Scala field currentMean  isn't represented as a Java field, neither it has a Java accessor method
note that private parameters of class constructors don't get mapped onto fields and/or accessors,
unless they are used outside of their declaring constructors.

scala version 2.11.7
java version 1.8.0_66

[xuwei-k]

Could not reproduce Scala 2.12.0 or later

[hrhino]

fixed by fields phase, maybe?

[som-snytt]

The example progressed because lambdas became static methods, so mangling the private accessor was no longer necessary.

However, as the bumper sticker says, "mangling happens":

class C {
  private val c = 42
}
object C {
  def f(c: C) = c.c
}

so

scala 2.13.1> import reflect.runtime._
import reflect.runtime._

scala 2.13.1> val c = new C
c: C = C@9f1ca74

scala 2.13.1> currentMirror.reflect(c).symbol.toType
res0: reflect.runtime.universe.Type = C

scala 2.13.1> .members
res1: reflect.runtime.universe.MemberScope =
Scope{
  private[this] val c: <?>;
  private val c: <?>;
  def <init>: <?>;
  final def $asInstanceOf[T0](): T0;
  final def $isInstanceOf[T0](): Boolean;
  final def synchronized[T0](x$1: T0): T0;
  final def ##(): Int;
  final def !=(x$1: Any): Boolean;
  final def ==(x$1: Any): Boolean;
  final def ne(x$1: AnyRef): Boolean;
  final def eq(x$1: AnyRef): Boolean;
  final def notifyAll(): Unit;
  final def notify(): Unit;
  protected[package lang] def clone(): Object;
  final def getClass(): Class[_];
  def hashCode(): Int;
  def toString(): String;
  def equals(x$1: Object): Boolean;
  final def wait(): Unit;
  final def wait(x$1: Long, x$2: Int): Unit;
  final def wait(x$1: Long): Unit;
  protected[package lang] def finalize(): Unit;
  fina...

scala 2.13.1> .collect { case m: universe.MethodSymbol if m.isGetter => m }
                                              ^
              warning: abstract type pattern reflect.runtime.universe.MethodSymbol is unchecked since it is eliminated by erasure
res2: Iterable[reflect.runtime.universe.MethodSymbol] = List(value c)

scala 2.13.1> currentMirror.reflect(c).reflectField(res2.head).get
scala.ScalaReflectionException: Scala field c  of class C isn't represented as a Java field, nor does it have a
Java accessor method. One common reason for this is that it may be a private class parameter
not used outside the primary constructor.
  at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$ErrorNonExistentField(JavaMirrors.scala:149)
  at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:278)
  at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:269)
  ... 28 elided

scala 2.13.1> :javap -pc C
[snip]
Compiled from "<pastie>"
public class C {
  private final int C$$c;

  public static int f(C);
    Code:
       0: getstatic     #19                 // Field C$.MODULE$:LC$;
       3: aload_0
       4: invokevirtual #21                 // Method C$.f:(LC;)I
       7: ireturn

  public int C$$c();
    Code:
       0: aload_0
       1: getfield      #24                 // Field C$$c:I
       4: ireturn

  public C();
    Code:
       0: aload_0
       1: invokespecial #30                 // Method java/lang/Object."<init>":()V
       4: aload_0
       5: bipush        42
       7: putfield      #24                 // Field C$$c:I
      10: return
}