Hadoop and Ammonite: "illegal cyclic reference involving object InterfaceAudience"

[tdyas]

I tried to get the Ammonite repl to work with the CDH 5.4 variant of Hadoop. Instantiating classes causes the exception pasted below about illegal cyclic reference involving object InterfaceAudience. InterfaceAudience is an annotation within the Hadoop project. The exception does not occur with the ordinary Scala repl.

Any ideas on how to further debug this?

(Scala 2.11.8 Java 1.8.0_92)
@ import org.apa
apache
@ import org.apache
final package apache
@ import org.apache.had
hadoop
@ import org.apache.hadoop
final package hadoop
@ import org.apache.hadoop.fs.Path
import org.apache.hadoop.fs.Path
@ val p = new Path("/user/tdyas")
scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving object InterfaceAudience
  scala.reflect.internal.Symbols$Symbol$$anonfun$info$3.apply(Symbols.scala:1502)
  scala.reflect.internal.Symbols$Symbol$$anonfun$info$3.apply(Symbols.scala:1500)
  scala.Function0$class.apply$mcV$sp(Function0.scala:34)
  scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
  scala.reflect.internal.Symbols$Symbol.lock(Symbols.scala:546)
  scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1500)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$coreLookup$1(JavaMirrors.scala:992)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$lookupClass$1(JavaMirrors.scala:998)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$classToScala1(JavaMirrors.scala:1003)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$toScala$1.apply(JavaMirrors.scala:97)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache$$anonfun$toScala$1.apply(TwoWayCaches.scala:38)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache.toScala(TwoWayCaches.scala:33)
  scala.reflect.runtime.JavaMirrors$JavaMirror.toScala(JavaMirrors.scala:95)
  scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy.<init>(JavaMirrors.scala:163)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
  scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
  scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
  scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$copyAnnotations(JavaMirrors.scala:683)
  scala.reflect.runtime.JavaMirrors$JavaMirror$FromJavaClassCompleter.load(JavaMirrors.scala:733)
  scala.reflect.runtime.JavaMirrors$JavaMirror$FromJavaClassCompleter.complete(JavaMirrors.scala:744)
  scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$coreLookup$1(JavaMirrors.scala:992)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$lookupClass$1(JavaMirrors.scala:998)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$classToScala1(JavaMirrors.scala:1003)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$toScala$1.apply(JavaMirrors.scala:97)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache$$anonfun$toScala$1.apply(TwoWayCaches.scala:38)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache.toScala(TwoWayCaches.scala:33)
  scala.reflect.runtime.JavaMirrors$JavaMirror.toScala(JavaMirrors.scala:95)
  scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy.<init>(JavaMirrors.scala:163)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
  scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
  scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
  scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$copyAnnotations(JavaMirrors.scala:683)
  scala.reflect.runtime.JavaMirrors$JavaMirror$FromJavaClassCompleter.load(JavaMirrors.scala:733)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:142)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:133)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:168)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.typeParams(SynchronizedSymbols.scala:132)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.typeParams(SynchronizedSymbols.scala:168)
  scala.reflect.internal.Types$class.isRawIfWithoutArgs(Types.scala:3756)
  scala.reflect.internal.SymbolTable.isRawIfWithoutArgs(SymbolTable.scala:16)
  scala.reflect.internal.Types$class.isRawType(Types.scala:3761)
  scala.reflect.internal.SymbolTable.isRawType(SymbolTable.scala:16)
  scala.reflect.internal.Types$class.normalizePlus(Types.scala:3933)
  scala.reflect.internal.SymbolTable.normalizePlus(SymbolTable.scala:16)
  scala.reflect.internal.tpe.TypeComparers$class.isSameType2(TypeComparers.scala:236)
  scala.reflect.internal.SymbolTable.isSameType2(SymbolTable.scala:16)
  scala.reflect.internal.tpe.TypeComparers$class.scala$reflect$internal$tpe$TypeComparers$$isSameType1(TypeComparers.scala:129)
  scala.reflect.internal.tpe.TypeComparers$class.isSameType(TypeComparers.scala:102)
  scala.reflect.internal.SymbolTable.isSameType(SymbolTable.scala:16)
  scala.reflect.internal.Types$Type.$eq$colon$eq(Types.scala:834)
  scala.reflect.internal.Types$Type.$eq$colon$eq(Types.scala:260)
  ammonite.repl.DefaultReplAPI$Internal$.print(ReplAPI.scala:233)

[lihaoyi]

No clue =( Maybe try asking on scala/contributors see if they have any tips? I imagine the compiler people must bump into this every so often and have some feel for what to look out for

[tdyas]

I dug in a little further and it turns out that the failing call is the typeOf call on line 233 of ReplAPI.scala. If I do typeOf[org.apache.hadoop.fs.Path] in the regular REPL, the same exception occurs. So definitely a Scala reflection issue and nothing Ammonite specific. I'll go inquire with the scala project as you suggest.

[tdyas]

Filed https://issues.scala-lang.org/browse/SI-10129 against Scala project

[tdyas]

Ammonite could avoid the typeOf by doing classOf[Unit].isAssignableFrom(value.getClass). Would you be okay with this workaround?

[lihaoyi]

Sure let's try that. There's only one Unit instance right? So value eq () should work just as well too.

If it fixes your problem locally send a PR and we'll merge it

[tdyas]

The change fixes the issue for me

PR: #543

[lihaoyi]

This should be fixed in the latest unstable version http://www.lihaoyi.com/Ammonite/#UnstableVersions