Latest merge with experimental fails

[tampler]

Type of issue: bug report

Two latest commits by Richard Lin on Jan21 break output generation.

[info] my_pkg.MaccTester *** ABORTED ***
[info] java.lang.NoClassDefFoundError: chisel3/core/ImplicitModule
[info] at chisel3.iotesters.setupTreadleBackend$.apply(TreadleBackend.scala:141)
[info] at chisel3.iotesters.Driver$.$anonfun$execute$2(Driver.scala:54)

This fails to compile, but release from Jan 18 works fine with my code here

Any support required ?

[jackkoenig]

This sounds like a binary incompatibility from #994. Can you provide more information about your flow? Are you publishing the projects locally yourself or pulling from the published snapshots?

In any case, this looks like your chisel-testers were compiled against a commit pre #994 but the chisel3 getting pulled in is post #994. You just need to make sure your testers are compiled against a newer local publish of chisel3.

If you're using the snapshots that are published on the web, then it sounds like there's an issue with our snapshot publishing.

[tampler]

Hi Jack

I'm pulling and building FIRRTL, Chisel3 and chisel-testers from master and use local snapshots.
Scala is 2.12.8 and sbt is 1.2.8

I get this error for all commits, starting from Jan21. I rolled back to Jan18 commits and all works seamlessly. I test with chisel_katas project

[albert-magyar]

As Jack says, it sounds like you need to re-run publishLocal in chisel-testers to get one that is compiled against latest chisel3.

However, the unfortunate truth is that it would be reasonable to try deleting your project classfiles to force the issue. I know I am not the only person to run into similar situations where sbt occasionally chokes.

[tampler]

I confirm that the problem persists. Check my log here: https://drive.google.com/open?id=1L4g1G39_g6VpVuSAw4mP4GahJcKa3eRZ

What I did:

1. Removed ~/.ivy2/local
2. make clean and git fetch from firrtl/ chisel3/ and chisel-iotesters/
3. git checkout master and git pull in all three
4. sbt publishLocal in all three

My test still fails for testOnly my_pkg.MaccTester -- -z Basic see the error log attached

[tampler]

same for commit eb6ddf4 on Jan 22 by Schuyler

[tampler]

@albert-magyar Removed all binary files from chisel-testers and rebuilt it from scratch. Didn't help as well. Something is wrong here

[ucbjrl]

@ducky64's PR renames ImplicitModule to MultiIOModule which explains why you'll get a:

[info] my_pkg.MaccTester *** ABORTED ***
[info]   java.lang.NoClassDefFoundError: chisel3/core/ImplicitModule

What's curious is why the stack trace claims:

[error] 	at chisel3.iotesters.setupTreadleBackend$.apply(TreadleBackend.scala:141)

It looks like when treadle support was added six months ago, the Module type parameter was:

  def apply[T <: MultiIOModule](

Make sure your code MaccTest.scala:21 isn't specifying something that generates an ImplicitModule as the dut argument to Driver.execute.

[albert-magyar]

I tried it out, and I am able to run testOnly my_pkg.MaccTester -- -z Basic using master on all the repos (firrtl, chisel3, chisel-testers, chisel_katas).

[tampler]

@albert-magyar Which OS are you running ? Hmm that's strange. I'll try that again tomorrow from scratch.
I had no issues with anything before Jan21.

I'm Running this on Ubuntu 16.04.5 with GCC8.1.0

bku@lap:~$ java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.16.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

[albert-magyar]

OS X 10.13.6

[ducky64]

I've had errors with sbt preferring sonatype versions over local versions even if local versions were published more recently. I'm not sure how to fix this, but this might be your problem, especially since your log says

[info] Updating ...
[warn] Resolving a snapshot version. It's going to be slow unless you use `updateOptions := updateOptions.value.withLatestSnapshots(false)` options.
[info] Out of 1 candidates we found for edu.berkeley.cs#chisel3_2.12;3.2-SNAPSHOT in sonatype-snapshots, we are choosing sonatype-snapshots.
[warn] Sorting results from edu.berkeley.cs#firrtl_2.12;1.2-SNAPSHOT, using Tue Jan 22 22:13:15 MSK 2019 and Tue Jan 22 22:13:15 MSK 2019.
[warn] Sorting results from edu.berkeley.cs#firrtl_2.12;1.2-SNAPSHOT, using Tue Jan 22 22:13:15 MSK 2019 and Tue Jan 22 22:13:15 MSK 2019.
[warn] Resolving a snapshot version. It's going to be slow unless you use `updateOptions := updateOptions.value.withLatestSnapshots(false)` options.
[info] Out of 2 candidates we found for edu.berkeley.cs#firrtl_2.12;1.2-SNAPSHOT in local and sonatype-snapshots, we are choosing local.
[info] Out of 2 candidates we found for edu.berkeley.cs#firrtl_2.12;1.2-SNAPSHOT in local and sonatype-snapshots, we are choosing sonatype-snapshots.
[warn] Resolving a snapshot version. It's going to be slow unless you use `updateOptions := updateOptions.value.withLatestSnapshots(false)` options.
[info] Out of 1 candidates we found for edu.berkeley.cs#chisel-iotesters_2.12;1.3-SNAPSHOT in sonatype-snapshots, we are choosing sonatype-snapshots.
[warn] Resolving a snapshot version. It's going to be slow unless you use `updateOptions := updateOptions.value.withLatestSnapshots(false)` options.
[info] Out of 1 candidates we found for edu.berkeley.cs#firrtl-interpreter_2.12;1.2-SNAPSHOT in sonatype-snapshots, we are choosing sonatype-snapshots.
[warn] Resolving a snapshot version. It's going to be slow unless you use `updateOptions := updateOptions.value.withLatestSnapshots(false)` options.
[info] Out of 1 candidates we found for edu.berkeley.cs#treadle_2.12;1.1-SNAPSHOT in sonatype-snapshots, we are choosing sonatype-snapshots.
[info] Done updating.

[ucbjrl]

Unfortunately, the sbt diagnostic output is incorrect for at least two reasons:

• for each candidate, it claims to be choosing that candidate (check out the sbt source code),
• it doesn't understand that after a successful publishLocal (at least on MacOS and Unix), what's in the .ivy2/cache/... directory isn't the actual Jar file, but a reference to the Jar file in the .ivy2/local/... directory.

[tampler]

@ucbjrl Jim, Can I rebuild treadle separately ? I didn't see a separate repo for it

If not possible, I'll clean cache and local in my ~/.ivy2 and will try from scratch.

[ucbjrl]

Yes. The treadle repo is https://github.com/freechipsproject/treadle.

[ucbjrl]

#998 has added aliases for UserModule and ImplicitModule so this should be resolved with a new pull. Still, it would be instructive to know why this was failing - who was referring to ImplicitModule?

[jackkoenig]

@ucbjrl, even if no one directly refers to ImplicitModule, isn't there a binary incompatibility between projects compiled against chisel3 where UserModule is an alias for ImplicitModule and those where ImplicitModule is renamed? I'm actually unsure if #998 fixes that problem.

[ucbjrl]

Possibly, but we're talking about SNAPSHOT (i.e., bleeding edge) versions, and I don't believe we currently make any guarantee about binary compatibility between SNAPSHOT releases.
We should discuss this at the next dev meeting.

The tacit assumption is that SNAPSHOT clients are compiling against SNAPSHOT versions.

[tampler]

The problem is now resolved. I removed ~/.ivy2/cache/edu.berkeley.cs folder, which indeed contained lots of stuff, including treadle.

Made a pull and rebuilt everything from scratch. Now works.

The question is how to maintain binary coherency with caching ? Should I remove all caches and the ~/.ivy2/local folder before publishing a fresh batch of snapshots ?

That was the first time I encountered such problem, so sorry for inconvenience

[tampler]

Thanks everybody for feedback