Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deadlock in repl using sys.process #9824

Closed
scabug opened this issue Jun 18, 2016 · 11 comments

Comments

@scabug
Copy link

commented Jun 18, 2016

Cay reported the following program that hangs in M4 (https://groups.google.com/forum/#!topic/scala-user/CMD_oZK_4Ns)

scala> import scala.sys.process._
import scala.sys.process._

scala> println(("ls -al /" #| "grep u").!!)

also hangs with -Ydelambdafy:inline (though with the library itself is compiled with delambdafy-method -- i did not try a scala library compiled with delambdafy-inline)

This also hangs in the class-based repl.


(The rest of the description duplicates #8119 parallel collections + object initialization = hang)

Also reported by cay horstmann on https://groups.google.com/forum/#!topic/scala-user/f5YJE5AB5aA

scala> for (i <- (1 to 100).par) yield i

hangs in 2.12.0-M4 and current master, thread dump below. it completes with -Ydelambdafy:inline, or by using -Yrepl-class-based.

thread dump:

scala> for (i <- (1 to 100).par) yield i

"scala-execution-context-global-18" #18 daemon prio=5 os_prio=31 tid=0x00007f90eb4fd800 nid=0x6a03 in Object.wait() [0x000070000571e000]
   java.lang.Thread.State: RUNNABLE
	at $line4.$read$$iw$$iw$$$Lambda$1143/1492822331.apply$mcII$sp(Unknown Source)
	at scala.runtime.java8.JFunction1$mcII$sp.apply(JFunction1$mcII$sp.java:12)
	at scala.collection.parallel.immutable.ParRange$ParRangeIterator.map2combiner(ParRange.scala:104)
	at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1052)
	at scala.collection.parallel.Task.scala$collection$parallel$Task$$$anonfun$1(Tasks.scala:49)
	at scala.collection.parallel.Task$$Lambda$1146/190975371.apply$mcV$sp(Unknown Source)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
	at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
	at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1049)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:156)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:146)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

[... more of those ...]

"process reaper" #10 daemon prio=10 os_prio=31 tid=0x00007f90eb4e7800 nid=0x5803 waiting on condition [0x0000700002a03000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000760008f60> (a java.util.concurrent.SynchronousQueue$TransferStack)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
	at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
	at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"main" #1 prio=5 os_prio=31 tid=0x00007f90ea00d000 nid=0x1703 in Object.wait() [0x0000700000617000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007a9ae3970> (a scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask)
	at java.util.concurrent.ForkJoinTask.externalAwaitDone(ForkJoinTask.java:334)
	- locked <0x00000007a9ae3970> (a scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask)
	at java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:391)
	at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:719)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:375)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
	at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult(Tasks.scala:423)
	at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
	at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult(Tasks.scala:555)
	at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
	at scala.collection.parallel.immutable.ParRange.map(ParRange.scala:34)
	at $line4.$read$$iw$$iw$.<init>(<console>:12)
	at $line4.$read$$iw$$iw$.<clinit>(<console>)
	at $line4.$eval$.$print$lzycompute(<console>:7)
	- locked <0x00000007a9abdbb0> (a $line4.$eval$)
	at $line4.$eval$.$print(<console>:6)
	at $line4.$eval.$print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:781)
	at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1042)
	at scala.tools.nsc.interpreter.IMain$WrappedRequest.scala$tools$nsc$interpreter$IMain$WrappedRequest$$$anonfun$54(IMain.scala:633)
	at scala.tools.nsc.interpreter.IMain$WrappedRequest$$Lambda$981/1862816278.apply(Unknown Source)
	at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
	at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:632)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:560)
	at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:804)
	at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:678)
	at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:393)
	at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:413)
	at scala.tools.nsc.interpreter.ILoop.scala$tools$nsc$interpreter$ILoop$$$anonfun$91(ILoop.scala:915)
	at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:901)
	at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:74)
	at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
	at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
	at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
	at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
@scabug

This comment has been minimized.

Copy link
Author

commented Jun 18, 2016

Imported From: https://issues.scala-lang.org/browse/SI-9824?orig=1
Reporter: @lrytz
Affected Versions: 2.12.0
See #9076, #8119
Duplicates #10007

@scabug

This comment has been minimized.

Copy link
Author

commented Jun 18, 2016

@retronym said:
Duplicate of #8119 ?

@scabug

This comment has been minimized.

Copy link
Author

commented Jun 18, 2016

@retronym said:
See also: #9076

@scabug

This comment has been minimized.

Copy link
Author

commented Jun 18, 2016

@lrytz said:
the parallel collections example passes with -Yrepl-class-based, so that's a duplicate of #8119.

@scabug

This comment has been minimized.

Copy link
Author

commented Aug 23, 2016

@SethTisue said:
regretfully postponed to 2.12.1. we should list as a known bug in the release notes.

@scabug

This comment has been minimized.

Copy link
Author

commented Oct 29, 2016

@som-snytt said:
The linked ticket includes a bug which is different from generic "threads + static init = deadlock"; also, the deadlock problem is somewhat more insidious with lambdas, apparently.

@scabug

This comment has been minimized.

Copy link
Author

commented Dec 15, 2016

@SethTisue said:
@adriaanm you put this in Backlog, but it's in the 2.12.0 release notes as a notable known bug...?

@scabug

This comment has been minimized.

Copy link
Author

commented Dec 15, 2016

@adriaanm said:
Looks like A.P.'s PR around sys.process actually fixed this in 2.12.1. There's a lot of things going on in these tickets, so I'm not sure anymore whether we can close this based on this evidence alone:

Welcome to Scala 2.12.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112).
Type in expressions for evaluation. Or try :help.

scala> import scala.sys.process._
import scala.sys.process._

scala>  println(("ls -al /" #| "grep u").!!)
drwxr-xr-x@   11 root  wheel    374 Nov  2 12:03 usr
// more stuff from my / ;-)
@scabug

This comment has been minimized.

Copy link
Author

commented Dec 15, 2016

@SethTisue said:
maybe it's #8119 we should still consider doing something about for 2.12.x, though. for (i <- (1 to 100).par) yield i still hangs. that's actually the hang I had considered release-note-worthy

@scabug

This comment has been minimized.

Copy link
Author

commented Dec 16, 2016

@som-snytt said (edited by @adriaanm on Dec 19, 2016 7:14:39 PM UTC):
Pretty sure this dupes #10007 and #9076 is the canonical REPL ticket, superseding #8119 (which is marred by par).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.