RunOnlyRegressionTest with executables that treat stdin like a tty fail on local scheduler

[sabo]

I wrote a small RunOnlyRegressionTest test that runs using the local scheduler and launcher, running a Python program. The program unexpectedly fails with this error output:

OSError: [Errno 25] Inappropriate ioctl for device

Running the command, as well as the job script created by ReFrame, does not cause the error. Neither does running the job script in a fresh python shell using subprocess.Popen.

I eventually tracked it back and was able to reproduce the error, with this as the executable

#!/usr/bin/env python3

import sys
import os
if sys.stdin.isatty():
    print("stdin is a tty")
    # do some tty-only thing
    print(os.tcgetpgrp(sys.stdin.fileno()))
else:
    print("stdin is not a tty")

and the Reframe test:

@rfm.simple_test
class TestBuggyStdin(rfm.RunOnlyRegressionTest):
    def __init__(self):
        self.descr = 'test buggy stdin'
        self.valid_systems = ['*']
        self.valid_prog_environs = ['*']
        self.executable = 'test.py'
        self.executable_opts = []
        self.sanity_patterns = sn.assert_found('stdin is not a tty', self.stdout)

rfm_TestBuggyStdin_job.out contains only stdin is a tty. The error file contains, as expected

Traceback (most recent call last):
  File "/verafs/home/joeuser/reframe_venv_3.8/bin/test.py", line 8, in <module>
    print(os.tcgetpgrp(sys.stdin.fileno()))
OSError: [Errno 25] Inappropriate ioctl for device

So, as far as I can tell, commands forked off by the local scheduler think that their stdin is a tty, when in reality it is not.
Just to rule out this being a python-only issue, I repeated the test with an equivalent Ruby script and got the same result.

Adding < /dev/null as an entry to executable_opts (overriding whatever stdin reframe is feeding to the script for the command in question) lets the tests run as expected.

Additionally, changing the call to run_command_async in reframe.core.schedulers.local.submit to pass either stdin=subprocess.PIPE or stdin=subprocess.DEVNULL to the underlying subprocess.Popen call makes things work.

[vkarak]

Thanks @sabo for spotting this and for doing the analysis! We have seen a similar error when we were trying to invoke Spack from within ReFrame in #1871, but we didn't have the time to investigate it, so thanks for doing that! :-) The fix in run_command_async seems to work fine as all unit tests pass, so we will apply it.