API Responses for Long Running Processes

[mikeschinkel]

@clubdeuce

One of the things I have been worried about with respect to running everything inside the box is for the user to get appropriate feedback for long-running processes such as Composer or anything else that generates many lines of output. It will not be a good UX if the developer has to wait until each long running command is complete for them to get any feedback on the command.

I thought to add an issue when I saw your work on src/class-cli-interface.php.

Recently I researched how to solve this problem and came across the use of the read command. It can take two forms, the first using a pipe and thus a sub-shell (meaning vars created inside the do ... done):

my-long-running-process | while read my_line; do
    echo "Line: ${my_line}"
done

Alternately this also performs the same without a sub-shell:

while read my_line; do
    echo "Line: ${my_line}"
done <<<  my-long-running-process 

So my thought is that our API calls could be called either as short-running or long-running as decided by the caller.

If called as long-running then the API would call and expect a token to be returned. This token could be managed using temp filenames as you see below (where mktemp creates a file replacing XXXXXX with the token value, e.g. /tmp/command.af47b3.log):

function output_filename() {
    local $output_file=$(mktemp /tmp/command.XXXXXX.log)
    touch $output_file
    echo $outputfile 
}

After generating the token the CLI could spawn a detached task and run the following (maybe using screen?):

$output_filename=$(output_filename) 
while read line; do
    echo "Line: ${line}" >> $output_filename
done <<<  $command
touch "${output_filename}.eof"

The first API call would simply return the token to the caller after kicking off the detached task above.

After the first call the API caller would call back with the token. The API would call this command with token and line_no passed in as arguments to return one or more lines of the output, whatever is currently available:

function read_output() {
    local token="$1"
    local line_no="$2"
    local command_output="/tmp/command.${token}.log"
    cat "${command_output}" | sed -n '${line_no},$ p'   #  -n=no-print, $=EOF, p=print
    if [ -f "${command_output}.eof" ]; then
        rm "${command_output}"
        rm "${command_output}.eof"
        return 1
    else
        return 0
    fi
}

On the third call the API caller would count the number of lines returned by the 2nd call and calculate the next line number. Obviously the API code in PHP would need to determine how many lines were returned each API call and the API might return to the caller (such as the Console or the Host CLI), something like this:

{
    "output" : "(Potentially) multi-line\noutput should go\nhere",
    "line_count" : 3,
    "next_line" : 4,
    "end_of_file": false
}

Does this make sense?