problem with run async=true & returncodes .....

[Maziar123]

Hi

below code always return [0, 0, None] and command never finish :
seems have problem with sleep ...

cmd = 'echo "Hello " && sleep 2  && echo "Finish " '
p = run(cmd, async_=True)

[vsajip]

You haven't specified any information about OS, etc. This script works for me, on Ubuntu 64-bit (Linux Mint):

import sarge

cmd = 'echo "Hello " && sleep 2  && echo "Finish " '
p = sarge.run(cmd, async_=True)
p.wait()  # wait for pipeline to finish
print(p.returncodes)

Running it prints:

Hello 
Finish 
[0, 0, 0]

So I plan to close this unless you can come up with more information and/or a small example demonstrating a problem.

[Maziar123]

Hi
Yes with p.wait work but when

import sarge

cmd = 'echo "Hello " && sleep 2  && echo "Finish " '
p = sarge.run(cmd, async_=True)


for i in range(1, 20):
    sleep(0.5)
    print('CMD:', p.commands, sep='')
    print('Count:', len(p.returncodes), ' ret:', p.returncodes, sep='')

I use it in async flow, then wait not suitable

Archlinux (lastest)
python 3.9
sarge .17

[vsajip]

I use it in async flow, then wait not suitable

Then you can use p.poll_all() or p.poll_last() to see if the pipeline has terminated, Example:

import sarge
import time

cmd = 'echo "Hello " && sleep 2  && echo "Finish " '
p = sarge.run(cmd, async_=True)

for i in range(1, 20):
    time.sleep(0.5)
    rcs = p.poll_all()
    if i == 1:
        print('cmd:', p.commands, sep='')
    print('%d: return codes: %s' % (i, rcs), sep='')
    if None not in rcs:
        break

prints

Hello 
cmd:[Command('echo Hello '), Command('sleep 2'), Command('echo Finish ')]
1: return codes: [0, 0, None]
Finish 
2: return codes: [0, 0, 0]

[Maziar123]

Thx ...

But you think behavior of returncodes is True ??

and with async_=True the "returncodes" in this sample without wait return none for infinite ??

[vsajip]

Yes, because the returncode is initially None and set to the actual return code when you poll or wait. That's just how subprocess works - see the Python docs for subprocess.Popen. As you can see from my example, the returncodes doesn't return None forever - the loop terminates.

[Maziar123]

the returncodes doesn't return None forever - the loop terminates.

then loop terminate with wrong "returncodes" ??

from API :
returncodes : A list of the return codes of all sub-processes which were actually run.

I think one of it is wrong : (one)API comment is wrong OR (TWO)returncodes behavior
and i think number TWO is wrong

[vsajip]

What exactly do you think is wrong? None is a placeholder for an actual return code.

[Maziar123]

in First example in async mode run it :
After 20 second returncodes remain [0, 0, None]

the command i think return with [0, 0, 0] after 2 second
without p.wait returncodes remain [0, 0, None] !!

[vsajip]

What happens if you use p.poll_all() or p.poll_last() as I suggested?

See this part of the documentation for subprocess, relating to the returncode attribute of subprocess.Popen objects.

[Maziar123]

as your comment :

Popen.returncode

The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated  yet.

focus in :

A None value indicates that the process hasn’t terminated yet

then sarge returncodes in async mode return wrong value because operation successfully finish ! seems your returncodes must fixed in async mode specially when have && and sleep

[vsajip]

It can also indicate that you haven't checked the status of the process - even if just using subprocess, returncode won't be set unless you call poll() or wait().

I just realised that the documentation for poll_all() and poll_wait() was missing. I have now updated the documentation to include them, as well as the documentation for returncodes. See here.

[vsajip]

If you are using async_ you can still call poll_all(), as it doesn't block.

[vsajip]

Two things need to happen for a returncode to be set:

1. The subprocess finishes
2. You call poll() or wait() on the corresponding subprocess.Popen object to get its returncode. These are called by Pipeline.poll_all() and Pipeline.wait() methods.

[Maziar123]

why returncodes internally not call poll_all specially in async mode ?

this do latest return state ...

[vsajip]

why returncodes internally not call poll_all specially in async mode ?

Generally in this kind of code you don't assume too much about what the caller wants, but provide mechanisms for them to do it. I could change it to call poll_all() internally, but then there's no way of decoupling the two things in your code, which might sometimes be desirable.

[vsajip]

I have no strong objection to changing it to poll the individual subprocesses, though.

[vsajip]

Well, I did it the way I did it because I wanted to let the users of the library decide when they want to do certain stuff. Your feedback is that you think it is too inconvenient to call poll_all(), and that the polling should be done automatically when you access returncodes. Having thought about it while we've been discussing the issue, I see that you have a reasonable point, so I will make the change to poll each subprocess for every access of returncodes.

[Maziar123]

clear code in better always :-)

[vsajip]

Thanks for helping to make sarge better 😄