New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hadoop and Ammonite: "illegal cyclic reference involving object InterfaceAudience" #535

Closed
tdyas opened this Issue Dec 22, 2016 · 7 comments

Comments

Projects
None yet
2 participants
@tdyas
Contributor

tdyas commented Dec 22, 2016

I tried to get the Ammonite repl to work with the CDH 5.4 variant of Hadoop. Instantiating classes causes the exception pasted below about illegal cyclic reference involving object InterfaceAudience. InterfaceAudience is an annotation within the Hadoop project. The exception does not occur with the ordinary Scala repl.

Any ideas on how to further debug this?

(Scala 2.11.8 Java 1.8.0_92)
@ import org.apa
apache
@ import org.apache
final package apache
@ import org.apache.had
hadoop
@ import org.apache.hadoop
final package hadoop
@ import org.apache.hadoop.fs.Path
import org.apache.hadoop.fs.Path
@ val p = new Path("/user/tdyas")
scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving object InterfaceAudience
  scala.reflect.internal.Symbols$Symbol$$anonfun$info$3.apply(Symbols.scala:1502)
  scala.reflect.internal.Symbols$Symbol$$anonfun$info$3.apply(Symbols.scala:1500)
  scala.Function0$class.apply$mcV$sp(Function0.scala:34)
  scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
  scala.reflect.internal.Symbols$Symbol.lock(Symbols.scala:546)
  scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1500)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$coreLookup$1(JavaMirrors.scala:992)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$lookupClass$1(JavaMirrors.scala:998)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$classToScala1(JavaMirrors.scala:1003)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$toScala$1.apply(JavaMirrors.scala:97)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache$$anonfun$toScala$1.apply(TwoWayCaches.scala:38)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache.toScala(TwoWayCaches.scala:33)
  scala.reflect.runtime.JavaMirrors$JavaMirror.toScala(JavaMirrors.scala:95)
  scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy.<init>(JavaMirrors.scala:163)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
  scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
  scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
  scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$copyAnnotations(JavaMirrors.scala:683)
  scala.reflect.runtime.JavaMirrors$JavaMirror$FromJavaClassCompleter.load(JavaMirrors.scala:733)
  scala.reflect.runtime.JavaMirrors$JavaMirror$FromJavaClassCompleter.complete(JavaMirrors.scala:744)
  scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$2.info(SynchronizedSymbols.scala:171)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$coreLookup$1(JavaMirrors.scala:992)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$lookupClass$1(JavaMirrors.scala:998)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$classToScala1(JavaMirrors.scala:1003)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToScala$1.apply(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$toScala$1.apply(JavaMirrors.scala:97)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache$$anonfun$toScala$1.apply(TwoWayCaches.scala:38)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.TwoWayCaches$TwoWayCache.toScala(TwoWayCaches.scala:33)
  scala.reflect.runtime.JavaMirrors$JavaMirror.toScala(JavaMirrors.scala:95)
  scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala(JavaMirrors.scala:980)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy.<init>(JavaMirrors.scala:163)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.reflect.runtime.JavaMirrors$JavaMirror$JavaAnnotationProxy$.apply(JavaMirrors.scala:162)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
  scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
  scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
  scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
  scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
  scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$copyAnnotations(JavaMirrors.scala:683)
  scala.reflect.runtime.JavaMirrors$JavaMirror$FromJavaClassCompleter.load(JavaMirrors.scala:733)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:142)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:133)
  scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
  scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:168)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.typeParams(SynchronizedSymbols.scala:132)
  scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.typeParams(SynchronizedSymbols.scala:168)
  scala.reflect.internal.Types$class.isRawIfWithoutArgs(Types.scala:3756)
  scala.reflect.internal.SymbolTable.isRawIfWithoutArgs(SymbolTable.scala:16)
  scala.reflect.internal.Types$class.isRawType(Types.scala:3761)
  scala.reflect.internal.SymbolTable.isRawType(SymbolTable.scala:16)
  scala.reflect.internal.Types$class.normalizePlus(Types.scala:3933)
  scala.reflect.internal.SymbolTable.normalizePlus(SymbolTable.scala:16)
  scala.reflect.internal.tpe.TypeComparers$class.isSameType2(TypeComparers.scala:236)
  scala.reflect.internal.SymbolTable.isSameType2(SymbolTable.scala:16)
  scala.reflect.internal.tpe.TypeComparers$class.scala$reflect$internal$tpe$TypeComparers$$isSameType1(TypeComparers.scala:129)
  scala.reflect.internal.tpe.TypeComparers$class.isSameType(TypeComparers.scala:102)
  scala.reflect.internal.SymbolTable.isSameType(SymbolTable.scala:16)
  scala.reflect.internal.Types$Type.$eq$colon$eq(Types.scala:834)
  scala.reflect.internal.Types$Type.$eq$colon$eq(Types.scala:260)
  ammonite.repl.DefaultReplAPI$Internal$.print(ReplAPI.scala:233)

@lihaoyi

This comment has been minimized.

Owner

lihaoyi commented Jan 1, 2017

No clue =( Maybe try asking on scala/contributors see if they have any tips? I imagine the compiler people must bump into this every so often and have some feel for what to look out for

@tdyas

This comment has been minimized.

Contributor

tdyas commented Jan 1, 2017

I dug in a little further and it turns out that the failing call is the typeOf call on line 233 of ReplAPI.scala. If I do typeOf[org.apache.hadoop.fs.Path] in the regular REPL, the same exception occurs. So definitely a Scala reflection issue and nothing Ammonite specific. I'll go inquire with the scala project as you suggest.

@tdyas

This comment has been minimized.

Contributor

tdyas commented Jan 1, 2017

Filed https://issues.scala-lang.org/browse/SI-10129 against Scala project

@tdyas

This comment has been minimized.

Contributor

tdyas commented Jan 1, 2017

Ammonite could avoid the typeOf by doing classOf[Unit].isAssignableFrom(value.getClass). Would you be okay with this workaround?

@lihaoyi

This comment has been minimized.

Owner

lihaoyi commented Jan 2, 2017

Sure let's try that. There's only one Unit instance right? So value eq () should work just as well too.

If it fixes your problem locally send a PR and we'll merge it

@tdyas

This comment has been minimized.

Contributor

tdyas commented Jan 4, 2017

The change fixes the issue for me

PR: #543

@lihaoyi

This comment has been minimized.

Owner

lihaoyi commented Jan 11, 2017

This should be fixed in the latest unstable version http://www.lihaoyi.com/Ammonite/#UnstableVersions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment