Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OS-Lib subprocess tests fail on Scala-Native 0.5.0 #3874

Closed
lihaoyi opened this issue Apr 15, 2024 · 1 comment · Fixed by #3875
Closed

OS-Lib subprocess tests fail on Scala-Native 0.5.0 #3874

lihaoyi opened this issue Apr 15, 2024 · 1 comment · Fixed by #3875

Comments

@lihaoyi
Copy link

lihaoyi commented Apr 15, 2024

They previously weren't supported at all on 0.4.x due to lack of multithreading. Now that multithreading exists, I would assume they should work, but they don't.

To repro,

  1. Check out this branch:Support os.proc on Scala Native com-lihaoyi/os-lib#257
  2. Run ./mill -i "os.jvm[2.12.17].test" test.os.SubprocessTests.lines to see the code passing on Scala-JVM
  3. Run ./mill -i "os.native[2.12.17].test" test.os.SubprocessTests.lines to see the exact same Scala code failing on Scala-Native 0.5.0

This gives me the following error:

----------------- Running Tests test.os.SubprocessTests.lines -----------------
X test.os.SubprocessTests.lines 70176ms
  java.lang.IllegalArgumentException
    scala.collection.immutable.VectorPointer.gotoNextBlockStartWritable(Unknown)
    scala.collection.immutable.VectorBuilder.gotoNextBlockStartWritable(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$eq(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$eq(Unknown)
    scala.collection.generic.Growable.$anonfun$$plus$plus$eq$1(Unknown)
    scala.collection.generic.Growable$$Lambda$1.apply(Unknown)
    scala.collection.Iterator.foreach(Unknown)
    scala.collection.AbstractIterator.foreach(Unknown)
    scala.collection.generic.Growable.$plus$plus$eq(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$plus$eq(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$plus$eq(Unknown)
    scala.collection.TraversableOnce.to(Unknown)
    scala.collection.AbstractIterator.to(Unknown)
    scala.collection.TraversableOnce.toIndexedSeq(Unknown)
    scala.collection.AbstractIterator.toIndexedSeq(Unknown)
    os.proc.call(Unknown)
    test.os.SubprocessTests$.$anonfun$tests$2(Unknown)

I'm not sure how to go about begin debugging this on Scala-Native, but presumably if the same code is passing on Scala-JVM and failing on Scala-Native, it is a Scala-Native bug.

At a first glance, it seems it might be some kind of multithreading bug, where we are trying to convert a java.util.concurrent.ConcurrentLinkedQueue into a Vector via a VectorBuilder, and someone along the chain is having its internal invariants violated

@lihaoyi
Copy link
Author

lihaoyi commented Apr 15, 2024

Here's a minimal repro that seems to hang in Scala Native 0.5.0

      val chunks = new java.util.concurrent.ConcurrentLinkedQueue[Int]

      chunks.add(1)

      println("spawn chunks " + chunks.size())
      println("spawn toIndexedSeq")
      import collection.JavaConverters._
      val chunksSeq = chunks.iterator.asScala.toIndexedSeq
      println(chunksSeq)

It hangs at the line val chunksSeq = .... On Scala-JVM it runs successfully

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant