Resource busy - /dev/ptyvf

[dblock]

This happens when you abort a test execution half way. It causes several issues.

• A ScriptAgent process will stay running forever and for some reason it cannot be killed. It's holding /dev/ptyvf, but it's not listed by lsof. Whoever finds out how to kill this thing via launchctl gets a 🍺 from me.
• Bwoken will hang because there's no stdout output, only stderr, which says Resource busy - /dev/ptyvf. Bwoken needs to pipe that properly so that the user sees it, and not hang.

[listrophy]

I find it helpful to do a ps aux | grep [r]uby to find still-running processes. Looking for ScriptAgent is also a good idea... hadn't thought of that!

The ruby processes are not always killable in random order... sometimes (if there are many), you have to find the right one to kill first. 😦

[dblock]

I think the problem is that this is launched under launchd, not that it's a Ruby process.

[dyba]

I've been having the same problem too. It may be necessary to check if /dev/ttyvf is also busy. The output of running lsof on my machine is:

ddyba@70ky0:~|⇒  lsof | grep ScriptAg
ScriptAge 4229 ddyba  cwd      DIR                1,1       646 1261393 /Users/ddyba/Code/Lectern
ScriptAge 4229 ddyba  txt      REG                1,1    203184  873277 /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator7.0.sdk/Developer/usr/bin/ScriptAgent
ScriptAge 4229 ddyba  txt      REG                1,1    600832   11333 /usr/lib/dyld
ScriptAge 4229 ddyba    0r     CHR                3,2       0t0     305 /dev/null
ScriptAge 4229 ddyba    1w     CHR              4,111     0t191     538 /dev/ttyvf
ScriptAge 4229 ddyba    2w     CHR              4,111     0t191     538 /dev/ttyvf
ScriptAge 4229 ddyba    5u  KQUEUE                                      count=3, state=0
ScriptAge 4229 ddyba    7u    unix 0xf6a619c28e4a8ccd       0t0         ->0xf6a619c28d046305
ScriptAge 4229 ddyba   10u    unix 0xf6a619c2a78d4245       0t0         ->0xf6a619c28e4a82a5
ScriptAge 4229 ddyba   12r     REG                1,1       804 1804951 /private/var/folders/fg/cx__jj0j4p5_b3sdj8hc94vr0000gn/T/instruments_simulator_7yb977
ScriptAge 4229 ddyba   14r     REG                1,1       804 1804951 /private/var/folders/fg/cx__jj0j4p5_b3sdj8hc94vr0000gn/T/instruments_simulator_7yb977
ScriptAge 4229 ddyba   15u    IPv4 0xf6a619c2a9b0e64d       0t0     TCP *:49786 (LISTEN)

ddyba@70ky0:~|⇒  ps aux | grep ScriptAgent
ddyba            4229   0.0  0.0   590380     24   ??  S     9:29AM   0:00.02 /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator7.0.sdk/Developer/usr/bin/ScriptAgent
ddyba            4923   0.0  0.0  2432784    604 s001  S+    9:48AM   0:00.00 grep ScriptAgent

[dyba]

I'd like to confirm that this indeed is being launched with launchd. Listing the files in the /tmp directory shows iphonesimulator sessions I have run on my machine:

com.apple.iphonesimulator.launchd.11aa3a72 com.apple.iphonesimulator.launchd.50222248 launch-S4mZV6
com.apple.iphonesimulator.launchd.1f539dc4 com.apple.iphonesimulator.launchd.50a453c4 launch-ep4hum
com.apple.iphonesimulator.launchd.20e0fa9a com.apple.iphonesimulator.launchd.5304b32b launch-xBy6yV
com.apple.iphonesimulator.launchd.23275ec2 com.apple.iphonesimulator.launchd.55f05fdd launchd-169.JV5Sl9
com.apple.iphonesimulator.launchd.255245c5 com.apple.iphonesimulator.launchd.5b502c36 launchd-170.VtNXmP
com.apple.iphonesimulator.launchd.2f852d7f com.apple.iphonesimulator.launchd.73593715 launchd-797.l6iW1X
com.apple.iphonesimulator.launchd.36c7de38 com.apple.iphonesimulator.launchd.8d1c49d  launchdadd-tool.PsyCLraY
com.apple.iphonesimulator.launchd.4a1f6906 com.apple.iphonesimulator.launchd.a86305c

[listrophy]

Killing the associated processes should let you move forward. Mind trying that?

I'm thinking about including some handling around this, but frankly, I'd rather Apple just fix the underlying problem so we don't have to even use tty/pty pairs. I've submitted a bug report, but there's been no movement on it.

[dyba]

What's the workaround for this issue? The only one I know of is restarting my machine, but I'd rather not do that if I can help it.

[listrophy]

The workaround is killing all the associated processes. Sometimes, you have to do it in a specific order, but I don't know the correct way to determine that order...

Otherwise, yeah. Restart. At the very least, try not to interrupt bwoken when it's running, and that should significantly reduce the restarts to practically zero. Of course, sometimes it'll hang and you have to interrupt anyway. 😢

[dblock]

I think I spent a day trying to find a way to find those processes to kill. Beer to whomever finds a repeatable script.

[dyba]

😞 Shutting down is really hampering my workflow. I've already had to shut down three times all due to bwoken hanging. @dblock did you make headway with identifying the processes that needed to be killed in order to get bwoken unstuck? I wasn't able to kill any of the associated processes successfully to get bwoken going again.

[listrophy]

I suppose you could just keep re-running this on the command line until all the offending processes are gone:

for proc in $(ps aux | grep -E "(bwoken|ScriptAgent)" | grep `whoami` | awk '{print $2}'); do for sig in 3 6 9; do kill -$sig $proc; done; done

/big-hammer

[dyba]

I'm going to bet the problem is with the unix_instruments.sh script. Perhaps, it would be a better approach to create a custom daemon using launchd? Apple Doc reference here Do you all agree or would such an approach be futile? I don't know too much about the history of this project and why that wrapper script was introduced or if any different approaches have been taken.

[listrophy]

unix_instruments.sh exists mainly because instruments(1) buffers the entirety of its output if you try to pipe it to another process. If you instead redirect the output, instruments(1) does not buffer the output and just streams it normally.

Without unix_instruments.sh (or the functionality it provides), you wouldn't get any output on the command line until instruments(1) exited.

I'd be happy to hear about other ways to get the output streamed, if that's possible. Otherwise, let Apple know about this at bugreport.apple.com.

[RobDay]

This is fixed here: https://gist.githubusercontent.com/jonathanpenn/1402258/raw/a208721479221b84fed42dc76340151b1d73957b/unix_instruments.sh

The original solution had potential to abandon the resource and the new solution replaces this with the usage of tee. Can this be incorporated into the main project?