Clean-up script to kill TagUI processes - eg if Ctrl+C is used to kill TagUI

[ck81]

@kensoh, as mentioned in #417, I pressed Ctrl-C a lot during testing and debugging of TagUI scripts.

In Windows 10, when I kill a TagUI script using Ctrl-C and immediately re-run the script, a lot of times I will get this warning message:

The process cannot access the file because it is being used by another process.

Although the error message appears, the script actually can still run properly. However, in the windows task bar at the bottom, I will see additional command windows that are leftover of the previous session (which are not closed properly).

I would like to write some scripts (using DOS) to delete away these "leftover" processes and command windows (as a result of pressing Ctrl-C). Want to do this because users are always worried when they see these error messages.

I noticed that when we run a tagui script, there is always a second command window that will also appear - that contains debugging as well as Sikuli info.

May I know in the error message:

The process cannot access the file because it is being used by another process

exactly which file is that?

Or if you have any suggestion how to kill those additional command windows which are leftover from the previous session?

p.s. to replicate this error message on Windows 10, simply run a TagUI script with some LIVE command in it. When the script pauses, kill the script using Ctrl-C. Then re-run the script again. Do this a couple of times and you will see the same error message, as well as more and more command window in the task bar.

[kensoh]

Hi CK, thanks for raising this and the detailed replication steps!

This has been resolved in #417 - where I created the scripts to kill dead processes (shared with you over email this solution on 2 May). Thereafter, the 2 scripts are committed in the cutting edge release.

For which file, it depends on which integrations are being called (eg SikuliX or just Chrome). There may also be other leftover files from breaking the script halfway during execution. TagUI has no access to clear as control is lost when Ctrl+C is used to prematurely terminate it.

macOS / Linux
https://github.com/kelaberetiv/TagUI/blob/master/src/end_processes

Windows
https://github.com/kelaberetiv/TagUI/blob/master/src/end_processes.cmd

[kensoh]

Adding on, besides trying to clean the dead processes due to Ctrl+C, what are the use cases for users having to break TagUI prematurely? Maybe there is some other ways to handle such use cases instead.

[ck81]

This has been resolved in #417 - where I created the scripts to kill dead processes (shared with you over email this solution on 2 May). Thereafter, the 2 scripts are committed in the cutting edge release.

I raised this issue again because the methods in #417 didn't work for me. I still continue to get the following errors on Windows 10

The process cannot access the file because it is being used by another process.

[ck81]

Adding on, besides trying to clean the dead processes due to Ctrl+C, what are the use cases for users having to break TagUI prematurely? Maybe there is some other ways to handle such use cases instead.

Such errors not only occur when users press Ctrl+C, it also occurs when there are errors in the TagUI script and the program stops with errors. The effect is the same as Ctrl-C, and when you re-run the script, you will see the same error:

The process cannot access the file because it is being used by another process

When developing a robot, we always make mistakes and such errors are very common.

[kensoh]

I see, for normal TagUI script error, can you give a replication scenario of error in TagUI script and this happens? It may be caused by another root cause that is not yet known.

Because the design is such that if errors happens in TagUI script, control will still be passed back to tagui.cmd batch file to do the cleaning up of background processes. But for Ctrl+C, control is lost immediately without being returned to tagui.cmd to do clean-up.

---

For Ctrl+C, I tried 10x with default PhantomJS mode without issues. For chrome option, I sometimes can get the process cannot access the file message. I notice when that happens when the Chrome integration process is still running and not closed by TagUI as Ctrl+C is used to kill it.

When I see the Chrome integration process command window still open after Ctrl+C, and I run end_processes.cmd, this window will close and when I run TagUI again there is no error message.

I can't replicate the case where running end_processes.cmd don't shut off that Chrome integration command window. Can you try restarting computer and each time after you Ctrl+C, run the end_processes.cmd? The best guess I have is maybe there are multiple dead processes in memory which the end_processes.cmd somehow only triggered closure of 1 but not all the dead processes killed before.

[kensoh]

Adding more technical notes for internal reference

After checking through TagUI's engines for handling integration processes (Chrome, Python, R, SikuliX), I found that only one of them has a potentially blocking code that could be the only point of weakness where the end_processes script can't kill. That is in tagui_chrome.php.

In that engine, when TagUI script has an action to interact with Chrome, TagUI sends an instruction to Chrome and then waits until Chrome responds back to acknowledge. This is a blocking transaction within the integration engine without timeout and is by design.

As it does not make sense to continue sending a next instruction when Chrome has not responded to the first one. In normal UI automation, sequential execution is very important and each step often has dependency on the success of the previous action.

while ($intent_result_string == "") {try {$intent_result_string = trim($client->receive());} catch (Exception $e) {}}

I couldn't imagine a scenario where Chrome does not respond. Otherwise, a breaker can be inserted above to check for the 'finish' signal from TagUI before looping. But doing so requires adding a sleep in order not to rapidly read from disk and consume even more CPU power.

However, that will result in slower performance for normal transactions between TagUI and Chrome. For eg, a flow with 10 actions may now take 10 seconds longer to complete, due to the additional overhead. Thus, without concrete evidence of benefit of inserting this breaker it would not be worth considering tackling from this angle.

[kensoh]

Though above finding and scenario is not replicable from my side, I think there should be a more robust end_process / end_processes.cmd that can kill any lingering processes from TagUI and friends. Will see if I can come up with something and update here again!

[kensoh]

Fyi CK, made a commit with following comments. Have a try! They are pretty aggressive and kill all running processes of TagUI and integration friends (SikuliX, Python, R, Chrome) and browser.

Scripts now available in cutting edge release - https://github.com/kelaberetiv/TagUI#set-up

Making a commit to improve on the existing script to kill TagUI processes.

For eg if Ctrl+C is used to kill TagUI prematurely, TagUI main process would lose the control to do clean-up of integrations (SikuliX, R, Python, Chrome) and Chrome browser.

This shell script (for all the 3 OSes) will kill those processes to free up memory and allows TagUI to start running from a clean state.

These actions are not done by default when TagUI is launched, as it is expected that TagUI exits cleanly. Otherwise cleaning up all processes by default will hide potential issues should they arise and they will end up not getting reported.

Below are contents of the scripts for Windows and macOS/Linux respectively -

end_processes.cmd

@echo off

rem set path to unix utilities for windows command prompt
if exist "%~dp0unx\gawk.exe" set "path=%~dp0unx;%path%"

:repeat_kill_php
for /f "tokens=* usebackq" %%p in (`wmic process where "caption like '%%php.exe%%' and commandline like '%%tagui_chrome.php%%'" get processid 2^>nul ^| cut -d" " -f 1 ^| sort -nur ^| head -n 1`) do set php_process_id=%%p
if not "%php_process_id%"=="" (
    taskkill /PID %php_process_id% /T /F > nul 2>&1
    goto repeat_kill_php
)

:repeat_kill_chrome
for /f "tokens=* usebackq" %%p in (`wmic process where "caption like '%%chrome.exe%%' and commandline like '%%tagui_user_profile --remote-debugging-port=9222%%'" get processid 2^>nul ^| cut -d" " -f 1 ^| sort -nur ^| head -n 1`) do set chrome_process_id=%%p
if not "%chrome_process_id%"=="" (
    taskkill /PID %chrome_process_id% /T /F > nul 2>&1
    goto repeat_kill_chrome
)

:repeat_kill_sikuli
for /f "tokens=* usebackq" %%p in (`wmic process where "commandline like '%%tagui.sikuli%%' and not caption like '%%wmic%%' and not caption like '%%cmd.exe%%'" get processid 2^>nul ^| cut -d" " -f 1 ^| sort -nur ^| head -n 1`) do set sikuli_process_id=%%p
if not "%sikuli_process_id%"=="" (
    taskkill /PID %sikuli_process_id% /T /F > nul 2>&1
    goto repeat_kill_sikuli
)

:repeat_kill_python
for /f "tokens=* usebackq" %%p in (`wmic process where "commandline like '%%tagui_py.py%%' and not caption like '%%wmic%%' and not caption like '%%cmd.exe%%'" get processid 2^>nul ^| cut -d" " -f 1 ^| sort -nur ^| head -n 1`) do set python_process_id=%%p
if not "%python_process_id%"=="" (
    taskkill /PID %python_process_id% /T /F > nul 2>&1
    goto repeat_kill_python
)

:repeat_kill_r
for /f "tokens=* usebackq" %%p in (`wmic process where "commandline like '%%tagui_r.R%%' and not caption like '%%wmic%%' and not caption like '%%cmd.exe%%'" get processid 2^>nul ^| cut -d" " -f 1 ^| sort -nur ^| head -n 1`) do set r_process_id=%%p
if not "%r_process_id%"=="" (
    taskkill /PID %r_process_id% /T /F > nul 2>&1
    goto repeat_kill_r
)

:repeat_kill_tagui
for /f "tokens=* usebackq" %%p in (`wmic process where "executablepath like '%%\\tagui\\src\\%%' and not caption like '%%cut.exe%%' and not caption like '%%sort.exe%%' and not caption like '%%head.exe%%'" get processid 2^>nul ^| cut -d" " -f 1 ^| sort -nur ^| head -n 1`) do set tagui_process_id=%%p
if not "%tagui_process_id%"=="" (
    taskkill /PID %tagui_process_id% /T /F > nul 2>&1
    goto repeat_kill_tagui
)

end_processes

#!/usr/bin/env bash

while true; do
    php_process_id="$(ps -x | grep tagui_chrome\.php | grep -v 'grep tagui_chrome\.php' | sed -e 's/^[ ]*//' | cut -d' ' -f 1 | sort -nur | head -n 1)"
    if [ -n "$php_process_id" ]; then
        kill $php_process_id > /dev/null 2>&1
    else
        break
    fi
done

while true; do
    chrome_process_id="$(ps -x | grep remote-debugging-port=9222 | grep tagui_user_profile | sed -e 's/^[ ]*//' | cut -d' ' -f 1 | sort -nur | head -n 1)"
    if [ -n "$chrome_process_id" ]; then
        kill $chrome_process_id > /dev/null 2>&1
    else
        break
    fi
done

while true; do
    sikuli_process_id="$(ps -x | grep tagui\.sikuli | grep -v 'grep tagui\.sikuli' | sed -e 's/^[ ]*//' | cut -d' ' -f 1 | sort -nur | head -n 1)"
    if [ -n "$sikuli_process_id" ]; then
        kill $sikuli_process_id > /dev/null 2>&1
    else
        break
    fi
done

while true; do
    python_process_id="$(ps -x | grep tagui_py\.py | grep -v 'grep tagui_py\.py' | sed -e 's/^[ ]*//' | cut -d' ' -f 1 | sort -nur | head -n 1)"
    if [ -n "$python_process_id" ]; then
        kill $python_process_id > /dev/null 2>&1
    else
        break
    fi
done

while true; do
    r_process_id="$(ps -x | grep tagui_r\.R | grep -v 'grep tagui_r\.R' | sed -e 's/^[ ]*//' | cut -d' ' -f 1 | sort -nur | head -n 1)"
    if [ -n "$r_process_id" ]; then
        kill $r_process_id > /dev/null 2>&1
    else
        break
    fi
done

while true; do
    tagui_process_id="$(ps -x | grep tagui/src | grep -v 'grep tagui/src' | grep -v 'end_processes' | sed -e 's/^[ ]*//' | cut -d' ' -f 1 | sort -nur | head -n 1)"
    if [ -n "$tagui_process_id" ]; then
        kill $tagui_process_id > /dev/null 2>&1
    else
        break
    fi
done

[kensoh]

Updated with a commit to filter away WMIC process on Windows to prevent hanging. Works now! 😓

[kensoh]

Committed further improvements to macOS and Linux end processes script above -

[kensoh]

Hopefully this aggressive kill script works for you now, CK. In addition, do let me know if you have replication steps where TagUI has below error message (due to dead processes), for situations where TagUI exits normally or run into errors and exit. Expected behaviour would be clean-up of background processes even if that happens, unless is killing by Ctrl+C or closing command prompt window.

The process cannot access the file because it is being used by another process

[ck81]

Thanks a lot, Ken!

Have just downloaded your latest release.

Trying it on my scripts. Will get back to you how it goes.

[ck81]

@kensoh, have pressed at least a hundred Ctrl-C's over the past 2 hours. Looks like you have fixed the thing!

Now I always see only one command window. There are no more leftover command prompts from the previous session like what I always have last time when I press Ctrl-C.

Thanks a lot, Ken. You're amazing!

[kensoh]

Thanks for your confirmation CK!!

I've iterated a few times over the last few days to improve on this kill processes script, but this sort of script to work across all OSes on an intimate level may encounter edge cases along the way. Closing the issue for now, but I'll look out for feedback from you and other users if there are new environmental setups and situations where this script can be improved further.

The reason I'm spending more time than I should on this is because I see the lack of ability to kill somehow dead TagUI processes as a roadblock for the tool to scale across enterprise. People will need to flush the environment now and then, and having this script allows fast flushing of background processes without restarting computer (and only takes 0.25 seconds on my 2014 laptop).

[kensoh]

(above iteration to remove warning message for end_processes script and tagui script on Linux OS)