Question: Printing status to stdout

[jkalleberg]

Question on: https://www.github.com/singularityhub/singularity-cli/tree/master/docs/pages/commands-images.md#execute

I'm able to load my image as a Client object. However, when I use Client.execute, I cannot keep the stdout as stdout. The command I'm trying to execute takes days to run, so having no progress or logging messages until failure occurs is a huge issue.

I've tried setting stream=True, quiet=False, and running Client.execute(....) instead of the output = Client.execute(...) in the docs. I've dug through the source code, and from what I can tell, I need to change capture=True to capture=False for sypython.utils.terminal.run_command. Is there a way to do this within the Client object? Please advise.

Note: I'm unable to use the spython shell as I'm working within a python script.

[vsoch]

I think if you want to both capture output lines (and print them as they come) from the Client you can do:

for line in Client.execute(container_sif, command_list, stream=True):
    print(line)

The above returns an iterator you can use to interact with output lines. Did this not work for some reason? It could be that it printing stdout only, and you need stderr. If you think this might be the case, we could expose an argument to control that.

[jkalleberg]

The above works, but only with quick commands in command_list, for example, ["/opt/command/bin/command", "--help"]. Otherwise, it appears to be waiting for the Client.execute(container_sif, command_list, stream=True) process to complete and does not return any stdout.

If I add quiet=False to get the singularity command used and then run the same command on the command line I immediately have the expected stdout on the terminal. Hence my confusion.

[vsoch]

Can you give me something to reproduce what you are seeing, and show me what you'd expect to have instead?

[jkalleberg]

It would take some time as I'm working with genomics tools and data. But, I'll put something together as quickly as possible.

[vsoch]

Sounds good! And no rush - I'd best be able to help after the work day. A simple repository with a container to pull (or build) and then whatever dummy assets (e.g., data) and then an example of what you are doing would be perfecto!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.