[WIP] Add macOS Terminal mark to tty prompts #11516

adamjstewart · 2019-05-19T21:00:33Z

Have you ever stared at the verbose output of a Spack build and lost track of where you were in the build process? Have you ever wished you could simply jump to the next stage in your terminal and see how the tests are progressing? With this PR, the future is now!

Simply use the macOS Terminal mark feature to jump between prompts using Cmd+Up/Down. See below .gif for a demonstration.

This works by calling the osascript command on macOS to insert a mark. Credits to Armin Briegel for figuring out how to do this.

lib/spack/llnl/util/tty/__init__.py

adamjstewart · 2019-05-19T21:02:34Z

lib/spack/llnl/util/tty/__init__.py

@@ -156,6 +162,10 @@ def msg(message, *args, **kwargs):


 def info(message, *args, **kwargs):


We should really combine tty.msg and tty.info. Does anyone know why they both exist?

lib/spack/llnl/util/tty/__init__.py

tgamblin · 2019-05-20T17:26:47Z

lib/spack/llnl/util/tty/__init__.py

@@ -24,6 +26,7 @@
 _warn_enabled = True
 _error_enabled = True
 indent = "  "
+_osascript = os.path.exists('/usr/bin/osascript')


Make this lazy (i.e., wrap it in a function call) so that it is not called unconditionally on module import. Doing lots of things in module scope slows down Spack startup.

tgamblin · 2019-05-20T17:29:02Z

lib/spack/llnl/util/tty/__init__.py

@@ -156,6 +164,11 @@ def msg(message, *args, **kwargs):


 def info(message, *args, **kwargs):
+    if _osascript:


Wrap this whole thing in a function so the code's not duplicated across all the subprocess.call invocations. And give the function a nice docstring to explain what's going on (I had to google to remember that osascript is the thing that runs AppleScripts)

tgamblin · 2019-05-20T17:45:11Z

lib/spack/llnl/util/tty/__init__.py

+    if _osascript:
+        subprocess.call(['osascript', '-e', 'if app "Terminal" is frontmost '
+                         'then tell app "System Events" to keystroke "u" '
+                         'using command down'])


Three things:

I am not sure that if app "Terminal" is frontmost is the best condition here. What if terminal is in the background while the output is spewing, but you want to ctrl-up after the fact? What if you're in xTerm or iTerm, but Terminal is in front (for whatever reason)?

What you really want to know is whether this spack instance was launched from the Terminal, and you can get that by checking the TERM_PROGRAM env var. In Apple Terminal it is set like this:

(spackbook):~$ env | grep TERM_PROGRAM TERM_PROGRAM=Apple_Terminal

Does iTerm support this too? In iTerm it is set like this:

(spackbook):~$ env | grep TERM_PROGRAM TERM_PROGRAM=iTerm.app

I think you can just check for those, then have the apple script unconditionally talk to "System Events".

The other thing that you likely need to factor in here is whether the output is going to stdout. Have you tried this with spack install without -v? I think you should maybe think about that, though maybe just doing it in msg and info is enough.

Are there markers that need to be suppressed with the terminal is not interactive? What if the user runs this and sends output to a file spack install -v hdf5 > log (people do this). You should probably only print out these markers sys.stdout.isatty() returns True.

The reason for if app "Terminal" is frontmost might be that it causes the Terminal to steal focus. I tried asking what the purpose was and got a response at the bottom of https://scriptingosx.com/2017/03/terminal-the-marks-the-spot/. I'll have to play around with this and confirm. I'll see what happens when the output is not verbose or is piped to a log.

Some observations:

Even if I remove if app "Terminal" is frontmost, it still doesn't add mark characters to the Terminal if I'm not currently focused on the Terminal. Luckily, it doesn't steal focus, but it does sound the terminal bell, which is annoying...

This doesn't seem to be an issue. I believe the osascript command is still run, and you can see that it adds a mark before the next ==> is displayed, but it doesn't break anything or add duplicate marks. Probably worth checking if verbose though.

This doesn't seem to be an issue. No meta characters are added to the log, although it has the same minor side-effect as 2.

I'm going to see if I can find a way to fix 1. It would be great if we could always add a mark character even if the Terminal is in the background. I also want to disable that bell...

Posted on StackOverflow: https://stackoverflow.com/questions/56262815/add-mark-to-terminal-in-background-using-applescript

I'm pretty sure that if you remove if app "Terminal" is frontmost then (i.e. only have tell app "System Events" to keystroke "u" using command down) then it will issue the keystroke to whatever app is currently active, causing who knows what. As a side note, I suspect that if one changes the shortcut then the AppleScript won't work—it might be more robust to script clicking the menu instead.

Another important test case will be what happens when multiple Terminal windows are open.

tldahlgren · 2019-05-20T17:53:24Z

lib/spack/llnl/util/tty/__init__.py

@@ -26,7 +26,7 @@
 _warn_enabled = True
 _error_enabled = True
 indent = "  "
-_osascript = os.path.exists('/usr/bin/osascript')
+_osascript = subprocess.call(['which', 'osascript']) == 0


Much cleaner. 👍

@adamjstewart @tldahlgren: only thing about this code is that it depends on which, and I kind of want to avoid having external command dependencies in this code.

The rest of spack uses spack.util.executable.which for things like this, but that returns a spack.util.executable.Executable, which is (kind of) spack-specific.

In #11245 I pulled out which_string from which(), that searches the path and gives you a str instead of an Executable. How about we pull that in as llnl.util.filesystem.which, and use it instead of shelling out to which?

Alternately, spack.util.executable is not THAT spack-specific. The only thing that you'd have to change to move it to llnl.util would be to rethink ProcessError, which extends SpackError, which a lot of parts of Spack use for error handling. Probably not worth dealing with that now -- but it's something we could do later. Just using which_string is probably fine for now.

We can't use which_string because spack.util.executable imports llnl.util.tty. We would either have to reimplement it in tty or remove the tty imports.

I meant move the function to llnl.util.

Are we still thinking about merging llnl and spack? Also, did you have any opinions on merging tty.msg and tty.info?

Not at the moment, though I have thought about making some things in llnl.util into proper Python packages.

I think you can merge info into msg. I have no recollection at this point of why they're different.

Did you want to keep info or msg? I'm leaning towards info since it's more inline with https://docs.python.org/3/library/logging.html#logging-levels. Also, should we have a command-line or config option to enable/disable marks?

Also, tty.verbose doesn't seem to be used anywhere? Is this correct?

adamjstewart · 2019-05-31T17:54:24Z

Another possible way to do things is to create a separate mark.scpt AppleScript file containing:

tell application "System Events"
    tell process "Terminal"
        click menu item "Mark" of menu "Marks" of menu item "Marks" of menu "Edit" of menu bar item "Edit" of menu bar 1
    end tell
end tell

Then the script can be called like so:

_script = os.path.join(os.path.dirname(__file__), 'mark.scpt')
subprocess.call(['osascript', _script])

This has the benefit that it doesn't send a keystroke to the current application, so it doesn't sound a bell when the application suddenly switches. However, it still doesn't add a mark if the Terminal is in the background.

adamjstewart · 2019-08-13T20:56:45Z

I just noticed that the python executable also adds Terminal marks on macOS, so there must be a reliable way to do this.

omor1 · 2019-08-14T03:29:33Z

I just noticed that the python executable also adds Terminal marks on macOS, so there must be a reliable way to do this.

That seems to be because of the "Automatically Mark Prompt Lines" option, which is enabled by default. I'm not sure what mechanism is used to detect prompt lines.

Add macOS Terminal mark to tty prompts

134df74

adamjstewart added feature macOS labels May 19, 2019

adamjstewart commented May 19, 2019

View reviewed changes

adamjstewart requested review from tgamblin and davydden May 19, 2019 21:20

Only execute if osascript exists

5179417

omor1 reviewed May 19, 2019

View reviewed changes

lib/spack/llnl/util/tty/__init__.py Show resolved Hide resolved

tldahlgren reviewed May 20, 2019

View reviewed changes

lib/spack/llnl/util/tty/__init__.py Outdated Show resolved Hide resolved

Check for osascript anywhere in the PATH

20f4148

tgamblin requested changes May 20, 2019

View reviewed changes

tldahlgren reviewed May 20, 2019

View reviewed changes

adamjstewart changed the title ~~Add macOS Terminal mark to tty prompts~~ [WIP] Add macOS Terminal mark to tty prompts May 22, 2019

adamjstewart added the WIP label May 22, 2019

tgamblin mentioned this pull request May 28, 2019

stage: make source_path available before stage is built #11528

Merged

adamjstewart closed this Apr 3, 2020

adamjstewart deleted the features/mark branch April 3, 2020 00:36

adamjstewart mentioned this pull request Apr 23, 2020

Show status information in terminal title #16259

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Add macOS Terminal mark to tty prompts #11516

[WIP] Add macOS Terminal mark to tty prompts #11516

adamjstewart commented May 19, 2019 •

edited

Loading

adamjstewart May 19, 2019

tgamblin May 20, 2019

tgamblin May 20, 2019 •

edited

Loading

tgamblin May 20, 2019

adamjstewart May 22, 2019

adamjstewart May 22, 2019

adamjstewart May 22, 2019

omor1 May 23, 2019

adamjstewart May 23, 2019

tldahlgren May 20, 2019

tgamblin May 20, 2019 •

edited

Loading

adamjstewart May 20, 2019

tgamblin May 21, 2019

adamjstewart May 21, 2019

tgamblin May 21, 2019

tgamblin May 21, 2019

adamjstewart May 22, 2019

adamjstewart May 22, 2019

adamjstewart commented May 31, 2019

adamjstewart commented Aug 13, 2019

omor1 commented Aug 14, 2019

		@@ -156,6 +162,10 @@ def msg(message, args, *kwargs):


		def info(message, args, *kwargs):

		@@ -156,6 +164,11 @@ def msg(message, args, *kwargs):


		def info(message, args, *kwargs):
		if _osascript:

[WIP] Add macOS Terminal mark to tty prompts #11516

[WIP] Add macOS Terminal mark to tty prompts #11516

Conversation

adamjstewart commented May 19, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tgamblin May 20, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tgamblin May 20, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adamjstewart commented May 31, 2019

adamjstewart commented Aug 13, 2019

omor1 commented Aug 14, 2019

adamjstewart commented May 19, 2019 •

edited

Loading

tgamblin May 20, 2019 •

edited

Loading

tgamblin May 20, 2019 •

edited

Loading