xonsh startup script fails on some platforms

[otakucode]

Hello all, I just found out about xonsh yesterday and have been checking it out. After installing it (cloned from github, then used pip install . in the cloned directory) my first discovery was that it wasn't put anywhere on my path. Then I found the 'xonsh' script in the scripts directory. Upon executing it, I get:

/usr/bin/env: python -u: No such file or directory

The problem is the shebang at the top of the script. It called /usr/bin/env python -u. Aside from the fact that 'python' on my system will get you Python 2.7 and Python 3 is called python3 (a common configuration when legacy and current Python are both installed), the real issue is that using parameters like that in a shebang doesn't work on many systems. After doing some reading, it appears that this would work OK on OSX but fail on most (if not all) Linux systems. The system processes "python -u" as a single string and looks for an executable with that whole name.

I'm not sure how you'd like this fixed. Personally I set the PYTHONUNBUFFERED environment variable to 1 before launching the script after changing the shebang to be #!/usr/bin/env python3. I don't think that's a good general answer, though. It would fail on systems which have python3 as just 'python' and not guarantee unbuffered access. Perhaps the disabling of buffering should be handled in code?

[rbrewer123]

I've been investigating along similar lines for improving xonsh startup on Windows. Currently we have scripts/xonsh.bat which occasionally asks me if I want to Terminate batch job? (y/n) when I exit xonsh for no apparent reason.

setup.py supports some console_script magic that generates a main script for unices with a shebang line localized to the current system; on Windows it creates a .exe that also acts as a main script. Unfortunately it doesn't support the -u flag for unbuffered startup, and it's not clear to me how to get PYTHONUNBUFFERED set for each invocation of xonsh but not every invocation of python. I don't yet know if console_script would have any undesirable interactions with the conda packaging.

Googling the unbuffered issue turned up some ugly ways of monkey patching sys.stdout and sys.stderr early in startup of xonsh but I was hoping for a nicer answer.

[adqm]

Can you try installing with sudo python3 setup.py install instead? That will translate the /usr/bin/env python -u to actually use the Python interpreter you install with, but with the -u flag.

I run GNU/Linux almost exclusively, and I have not had a problem at all when installing things that way (not sure whether installing from pypi with pip or installing from conda.

[scopatz]

I think that we could safely change the python executable to python3.

[scopatz]

Also, I am for whatever changes need to be made to the xonsh.bat script too.

[melund]

For the conda packaging it is enough to add the following to the build recipe:

build:  
  entry_points:
    - xonsh = xonsh.main:main

The conda build machinery will create a xonsh.exe file that points to a xonsh-script.py file in the script directory.

if __name__ == '__main__':
    import sys
    from xonsh.main import main

    sys.exit(main())

It just overrides any script defined in setup.py, so there is no problem with adding this.

[rbrewer123]

I like this idea, but my concern is losing the unbuffered mode mentioned above. Is there a way to enable unbuffered mode with this method?

[melund]

Hi @rbrewer123 .... I don't know. Are you sure that it is not already enabled? Can you give me a simple way to test it?

[lmmx]

A thread that may be of interest: StackOverflow: disable output buffering

     class Unbuffered(object):
        def __init__(self, stream):
            self.stream = stream
        def write(self, data):
            self.stream.write(data)
            self.stream.flush()
        def __getattr__(self, attr):
            return getattr(self.stream, attr)

     import sys
     sys.stdout = Unbuffered(sys.stdout)

or

# reopen stdout file descriptor with write mode
# and 0 as the buffer size (unbuffered)
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

More simply, you could just make it a shell script. This is what I'm now using on Linux:

#!/usr/bin/env sh
export PYTHONUNBUFFERED="XONSH_SET"
/usr/bin/env python -c "from xonsh.main import main; main()"

/usr/bin/env python -uc would also work in this context, but maybe the above is clearer?

I'm not sure what best practice is but in e.g. Perl -w was deprecated for explicit "use warnings". As someone said already, -u doesn't work on Linux unless installing with sudo. Here an environment variable seems the more robust way of setting unbuffered (the previous 2 examples seem quite awkward, and don't seem to unbuffer STDERR)...

[scopatz]

Thanks for the pointer @lmmx - if anyone has a PR I am happy to review & merge. I have never experienced this issue, so lacking a way to test, I am not the best person to propose a fix.

[lmmx]

The output on Linux Mint 17 (Ubuntu 14.04) for the xonsh command (scripts/xonsh) at present was

/usr/bin/env: python -u: No such file or directory

Apparently sudo would interpret the flag, but obviously that's not appropriate. I'll submit a PR now. Went for the explicit environment variable as it's clearer (NB it doesn't persist after quitting xonsh). The string has to be non-empty so I use "XONSH_SET" rather than "true" to distinguish against anything else that might set it.

[scopatz]

I am curious as to why sudo comes into this at all. Are you running sudo xonsh? You shouldn't need to do that...

[gforsyth]

I'm closing this issue as it seems to be resolved, but please reopen if that isn't the case.