Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does it work under Windows/Cygwin? #514

Closed
tkossak opened this issue Nov 19, 2015 · 78 comments
Closed

Does it work under Windows/Cygwin? #514

tkossak opened this issue Nov 19, 2015 · 78 comments
Assignees
Labels
Milestone

Comments

@tkossak
Copy link

tkossak commented Nov 19, 2015

Windows 7 x64
Cygwin x86
Python3 3.4.3

I installed xonsh with pip3 install xonsh and when I try to run it with xonsh I get the error:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.4/site-packages/xonsh/readline_shell.py", line 281, in run
    RL_LIB.history_set_pos(i)
  File "/usr/lib/python3.4/ctypes/__init__.py", line 364, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.4/ctypes/__init__.py", line 369, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'history_set_pos' not found (�[���%��P���)
@scopatz
Copy link
Member

scopatz commented Nov 19, 2015

You may need to install gnureadline or use prompt-toolkit instead.

@scopatz scopatz added the bug label Nov 19, 2015
@scopatz scopatz added this to the v0.3 milestone Nov 19, 2015
@scopatz
Copy link
Member

scopatz commented Nov 19, 2015

Thanks for reporting @tkossak

@tkossak
Copy link
Author

tkossak commented Nov 20, 2015

pip3 install gnureadline gave me errors (gcc can't find lncurses) but I
managed to run xonsh with prompt-toolkit. The shell starts with no errors, but
when I run any shell command (ie: ls, vim, echo, etc.) I get the error:

Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
    self.execer.exec(code, mode='single', glbs=self.ctx)  # no locals
  File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
    return exec(code, glbs, locs)
  File "<xonsh-code>", line 1, in <module>
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
    return run_subproc(cmds, captured=False)
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
    wait_for_active_job()
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
    _give_terminal_to(pgrp)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
    oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'

@scopatz
Copy link
Member

scopatz commented Nov 20, 2015

What terminal emulator are you running in? Also are you using cygwin's Python? Also, what is the output of:

import xonsh.tools
print(xonsh.tools.ON_WINDOWS, xonsh.tools.ON_LINUX, xonsh.tools.ON_POSIX)

I bet this is xonsh becoming confused about whether you are on windows or posix.

@scopatz
Copy link
Member

scopatz commented Nov 20, 2015

This might be the first report of anyone running in cygwin.

@tkossak
Copy link
Author

tkossak commented Nov 20, 2015

hehe, I'm forced to use windows at work so the least I can do to make it behave like real os is installing cygwin :)

My terminal emulator: mintty + tmux (and everything inside Cmder/ConEmu)
I'm using cygwin's python/pip, not window's ones. In bash:

$ which pip pip2 pip3 python python2 python3
/usr/bin/pip
/usr/bin/pip2
/usr/bin/pip3
/usr/bin/python
/usr/bin/python2
/usr/bin/python3

$ pip --version
pip 7.1.2 from /usr/lib/python2.7/site-packages (python 2.7)
$ pip2 --version
pip 7.1.2 from /usr/lib/python2.7/site-packages (python 2.7)
$ pip3 --version
pip 7.1.2 from /usr/lib/python3.4/site-packages (python 3.4)
$ python --version
Python 2.7.10
$ python2 --version
Python 2.7.10
$ python3 --version
Python 3.4.3

In xonsh:

$ which pip pip2 pip3 python python2 python3
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
self.execer.exec(code, mode='single', glbs=self.ctx) # no locals
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
return exec(code, glbs, locs)
File "<xonsh-code>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
return run_subproc(cmds, captured=False)
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
wait_for_active_job()
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
_give_terminal_to(pgrp)
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'
/usr/bin/pip
/usr/bin/pip2
/usr/bin/pip3
/usr/bin/python
/usr/bin/python2
/usr/bin/python3


$ pip --version
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
self.execer.exec(code, mode='single', glbs=self.ctx) # no locals
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
return exec(code, glbs, locs)
File "<xonsh-code>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
return run_subproc(cmds, captured=False)
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
wait_for_active_job()
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
_give_terminal_to(pgrp)
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'
pip 7.1.2 from /usr/lib/python2.7/site-packages (python 2.7)

$ pip2 --version
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
self.execer.exec(code, mode='single', glbs=self.ctx) # no locals
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
return exec(code, glbs, locs)
File "<xonsh-code>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
return run_subproc(cmds, captured=False)
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
wait_for_active_job()
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
_give_terminal_to(pgrp)
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'
pip 7.1.2 from /usr/lib/python2.7/site-packages (python 2.7)

$ pip3 --version
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
self.execer.exec(code, mode='single', glbs=self.ctx) # no locals
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
return exec(code, glbs, locs)
File "<xonsh-code>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
return run_subproc(cmds, captured=False)
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
wait_for_active_job()
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
_give_terminal_to(pgrp)
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'
pip 7.1.2 from /usr/lib/python3.4/site-packages (python 3.4)

$ python --version
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
self.execer.exec(code, mode='single', glbs=self.ctx) # no locals
Python 2.7.10
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
return exec(code, glbs, locs)
File "<xonsh-code>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
return run_subproc(cmds, captured=False)
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
wait_for_active_job()
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
_give_terminal_to(pgrp)
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'

$ python2 --version
Traceback (most recent call last):
Python 2.7.10
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
self.execer.exec(code, mode='single', glbs=self.ctx) # no locals
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
return exec(code, glbs, locs)
File "<xonsh-code>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
return run_subproc(cmds, captured=False)
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
wait_for_active_job()
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
_give_terminal_to(pgrp)
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'

$ python3 --version
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 126, in default
self.execer.exec(code, mode='single', glbs=self.ctx) # no locals
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
return exec(code, glbs, locs)
File "<xonsh-code>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 636, in subproc_uncaptured
return run_subproc(cmds, captured=False)
File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 606, in run_subproc
wait_for_active_job()
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 110, in wait_for_active_job
_give_terminal_to(pgrp)
File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 88, in _give_terminal_to
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'
Python 3.4.3

and the code you asked inside xonsh:

$ import xonsh.tools
$ print(xonsh.tools.ON_WINDOWS, xonsh.tools.ON_LINUX, xonsh.tools.ON_POSIX)
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 158, in push
locs=self.ctx)
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 94, in compile
tree = self.parse(input, ctx, mode=mode)
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 69, in parse
tree = self._parse_ctx_free(input, mode=mode)
File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 150, in _parse_ctx_free
debug_level=self.debug_level)
File "/usr/lib/python3.4/site-packages/xonsh/parser.py", line 263, in parse
tree = self.parser.parse(input=s, lexer=self.lexer, debug=debug_level)
File "/usr/lib/python3.4/site-packages/ply/yacc.py", line 331, in parse
return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
File "/usr/lib/python3.4/site-packages/ply/yacc.py", line 1106, in parseopt_notrack
p.callable(pslice)
File "/usr/lib/python3.4/site-packages/xonsh/parser.py", line 2098, in p_arglist
for arg in p1:
TypeError: 'Attribute' object is not iterable

@scopatz
Copy link
Member

scopatz commented Nov 21, 2015

What version of ply are you on? Are you sure you aren't running xonsh in python 2?

@tkossak
Copy link
Author

tkossak commented Nov 21, 2015

ran inside xonsh:

$ import sys
$ print(sys.version)
3.4.3 (default, May  5 2015, 17:04:32)
[GCC 4.9.2]
$ import ply
$ ply.__version__
'3.7'

@scopatz
Copy link
Member

scopatz commented Nov 21, 2015

OK. I have no idea what is going on here. Can you try asking on the mailing list or maybe stack overflow. Unfortunately, I don't have access to a windows system to test things out myself. But maybe other xonsh-windows users could try it out on cygwin.

@tildebyte
Copy link

I don't get any errors; I just don't get anything. I run xonsh, and I get nothing - no prompt, no exit back to bash.

CTRL-C to exit gives me a traceback.

RE: "gnureadline or use prompt-toolkit instead" - I installed prompt-toolkit, but I have no idea what xonsh is using.

Latest (as of today) cygwin x64.

  • "Pure" Cygwin, i.e. gnome-terminal 3.18.2 on Cyg-X.
  • Windows 7 Enterprise sp1.
  • Python 3.4.3 virtual environment.
(xonsh) foo@foo.com ~/
$ python
Python 3.4.3 (default, May  5 2015, 17:58:45) 
[GCC 4.9.2] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import xonsh.tools
>>> print(xonsh.tools.ON_WINDOWS, xonsh.tools.ON_LINUX, xonsh.tools.ON_POSIX)
False False True
>>> import ply
>>> ply.__version__
'3.7'

As far as xonsh being confused about the platform it's running on, this is the xonsh command I ended up with. Interesting.

@scopatz
Copy link
Member

scopatz commented Feb 23, 2016

Hi @tildebyte - thanks for reporting. Can you run the xonfig command and report back what it tells you? This is super weird. If you need to run it manually, you should be able to do:

from xonsh import xonfig
xonfig.main([])

@tildebyte
Copy link

Python 3.4.3 (default, May  5 2015, 17:58:45) 
[GCC 4.9.2] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from xonsh import xonfig
>>> xonfig.main([])
+------------------+-------+
| xonsh            | 0.2.6 |
| Python           | 3.4.3 |
| PLY              | 3.7   |
| have readline    | True  |
| prompt toolkit   | 0.57  |
| pygments         | 2.1.1 |
| on posix         | True  |
| on linux         | False |
| on arch          | False |
| on windows       | False |
| on mac           | False |
| are root user    | False |
| default encoding | utf-8 |
+------------------+-------+

@scopatz
Copy link
Member

scopatz commented Feb 27, 2016

Hi @tildebyte - OK it looks like it is getting stuck in the foreign shell phase on startup. Can you try adding a config file that does not load in any foreign shells, as described http://xon.sh/xonshconfig.html

Basically make ~/.config/xonsh/config.json look like:

{}

We should probably remove bash from the default foreign shells on posix.

@tildebyte
Copy link

Hmm. Neither {} nor (as from the docs) {"foreign_shells": []} help.

@scopatz
Copy link
Member

scopatz commented Mar 1, 2016

Hmm. Is your $BASH_COMPLETIONS variable set to an empty list?

@scopatz
Copy link
Member

scopatz commented Mar 1, 2016

or tuple.

@tildebyte
Copy link

$BASH_COMPLETIONS is unset.

@scopatz
Copy link
Member

scopatz commented Mar 1, 2016

@tkossak could you try running xonsh through a debugger or profiler to see where the problem is coming from?

@scopatz
Copy link
Member

scopatz commented Mar 1, 2016

I feel like the error is probably something trivial if we could identify what it is.

@tkossak
Copy link
Author

tkossak commented Mar 1, 2016

When I checked xonsh today I have the same results as @tildebyte - after running xonsh i get no errors/no prompt/no exit back to bash. When i press CTRL+C i get:

Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.4/trace.py", line 858, in <module>
    main()
  File "/usr/lib/python3.4/trace.py", line 804, in main
    t.runctx(code, globs, globs)
  File "/usr/lib/python3.4/trace.py", line 510, in runctx
    exec(cmd, globals, locals)
  File "/usr/bin/xonsh", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python3.4/site-packages/xonsh/main.py", line 166, in main
    args = premain(argv)
  File "/usr/lib/python3.4/site-packages/xonsh/main.py", line 153, in premain
    shell = builtins.__xonsh_shell__ = Shell(**shell_kwargs)
  File "/usr/lib/python3.4/site-packages/xonsh/shell.py", line 61, in __init__
    self._init_environ(ctx, config, rc)
  File "/usr/lib/python3.4/site-packages/xonsh/shell.py", line 99, in _init_environ
    self.execer = Execer(config=config)
  File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 40, in __init__
    load_builtins(execer=self, config=config)
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 675, in load_builtins
    builtins.__xonsh_env__ = ENV = Env(default_env(config=config))
  File "/usr/lib/python3.4/site-packages/xonsh/environ.py", line 1099, in default_env
    issue_warning=False))
  File "/usr/lib/python3.4/site-packages/xonsh/foreign_shells.py", line 419, in load_foreign_envs
    shenv, _ = foreign_shell_data(**shell)
  File "/usr/lib/python3.4/functools.py", line 472, in wrapper
    result = user_function(*args, **kwds)
  File "/usr/lib/python3.4/site-packages/xonsh/foreign_shells.py", line 173, in foreign_shell_data
    universal_newlines=True)
  File "/usr/lib/python3.4/subprocess.py", line 609, in check_output
    output, unused_err = process.communicate(inputdata, timeout=timeout)
  File "/usr/lib/python3.4/subprocess.py", line 960, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.4/subprocess.py", line 1617, in _communicate
    ready = selector.select(timeout)
  File "/usr/lib/python3.4/selectors.py", line 367, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

xonfig.main([]) returns:

Python 3.4.3 (default, May  5 2015, 17:58:45)
[GCC 4.9.2] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from xonsh import xonfig
>>> xonfig.main([])
'+------------------+-------+
| xonsh            | 0.2.6 |
| Python           | 3.4.3 |
| PLY              | 3.7   |
| have readline    | True  |
| prompt toolkit   | False |
| pygments         | 2.0.2 |
| on posix         | True  |
| on linux         | False |
| on arch          | False |
| on windows       | False |
| on mac           | False |
| are root user    | False |
| default encoding | utf-8 |
+------------------+-------+
'
>>>

I tried running trace (using python trace module with --trace option) and I got this 12 MB file: https://gist.github.com/tkossak/3705a380664e33439385

Everything was run in Cygwin x64 (if it matters at all).

Hope it helps.

@scopatz
Copy link
Member

scopatz commented Mar 1, 2016

Ok, not sure if this will do anything, but maybe you could try setting $FORCE_POSIX_PATHS = True if you haven't already.

@scopatz
Copy link
Member

scopatz commented Mar 1, 2016

This seems to be getting hung up on running path manipulation while running subprocesses. In particular, it is not happy with:

environ.py(125):     xcd = os.path.join(env.get('XDG_CONFIG_HOME'), 'xonsh')

@tkossak
Copy link
Author

tkossak commented Mar 1, 2016

Running export FORCE_POSIX_PATHS=True before running xonsh didn't change anything.
I don't have XDG_CONFIG_HOME environmental variable but i tried export XDG_CONFIG_HOME=/home/<USER>/.config with no luck either.

@Terrance
Copy link
Contributor

I'm in a similar position here, though I don't seem to be getting very far into the startup. Running xonsh shows no output. Running via verbose Python:

$ python -v
...
Python 3.4.3 (default, May  5 2015, 17:58:45)
[GCC 4.9.2] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from xonsh import main
...
>>> main.main()
# /usr/lib/python3.4/site-packages/xonsh/__pycache__/parser_table.cpython-34.pyc matches /usr/lib/python3.4/site-packages/xonsh/parser_table.py
# code object from '/usr/lib/python3.4/site-packages/xonsh/__pycache__/parser_table.cpython-34.pyc'
import 'xonsh.parser_table' # <_frozen_importlib.SourceFileLoader object at 0x6ffff129b38>

At that point it just hangs indefinitely. I have tried the suggestions above, but I suspect this isn't getting far enough for those to be the problem.

However, piping to stdin seems to bring me up to the error noted in the previous comments (note commands are being run, the output ending up amongst the stack trace):

$ xonsh <<EOF
ls
EOF
/usr/lib/python3.4/site-packages/xonsh/shell.py:97: RuntimeWarning: prompt-toolkit version < v0.57 and may not work as expected. Please update.
  warn(msg, RuntimeWarning)
Traceback (most recent call last):
  File "/usr/bin/xonsh", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python3.4/site-packages/xonsh/main.py", line 205, in main
    shell.execer.exec(code, mode='exec', glbs=shell.ctx)
  File "/usr/lib/python3.4/site-packages/xonsh/execer.py", line 126, in exec
    return exec(code, glbs, locs)
  File "<stdin>", line 1, in <module>
[output of ls appears here]
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 726, in subproc_uncaptured
    return run_subproc(cmds, captured=False)
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 653, in run_subproc
    wait_for_active_job()
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 113, in wait_for_active_job
    _give_terminal_to(pgrp)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 82, in _give_terminal_to
    oldmask = signal.pthread_sigmask(signal.SIG_BLOCK, _block_when_giving)
AttributeError: 'module' object has no attribute 'pthread_sigmask'

Output from xonfig:

$ python
Python 3.4.3 (default, May  5 2015, 17:58:45)
[GCC 4.9.2] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from xonsh import xonfig
>>> print(xonfig.main([]))
+------------------+-------+
| xonsh            | 0.2.7 |
| Python           | 3.4.3 |
| PLY              | 3.7   |
| have readline    | True  |
| prompt toolkit   | 1.0.0 |
| pygments         | 2.1.3 |
| on posix         | True  |
| on linux         | False |
| on arch          | False |
| on windows       | False |
| on mac           | False |
| are root user    | False |
| default encoding | utf-8 |
+------------------+-------+

@scopatz
Copy link
Member

scopatz commented May 13, 2016

what happens if you do exactly,

import xonsh.tools
xonsh.tools.ON_WINDOWS = False

in the xonshrc file?

@scopatz
Copy link
Member

scopatz commented May 13, 2016

I think that the problem is that it is picking up the Windows subprocess model, when it should get the UNIX model since you are on Cygwin

@scopatz
Copy link
Member

scopatz commented May 13, 2016

also is there a reliable way to detect we are on cygwin?

@Terrance
Copy link
Contributor

Using the xonshrc doesn't seem to make a difference (xonfig is already detecting posix though?). I should also confirm I get a similar Ctrl-C traceback to the one in this comment, regardless of xonshrc.

As for detecting Cygwin, we have these:

>>> sys.platform
'cygwin'
>>> platform.system()
'CYGWIN_NT-10.0'

The latter varies by Windows version and 32/64 variant, but appears to always begin with CYGWIN_NT-.

@ellocogato
Copy link

@adqm I have recently updated my Cygwin install. DLL version is 2.5.1.

@adqm
Copy link
Member

adqm commented Jun 8, 2016

@ellocogato, and you're still seeing that ProcessLookupError?

@ellocogato
Copy link

ellocogato commented Jun 8, 2016

@adqm Yes, I still see it. I am currently setting up a clean Cygwin environment on a different machine to test xonsh, and I'll report whether that issue shows up there as well.

Edit: I finished the setup on the other machine, and the experience is the same. Still getting the delay (if anything, it's even longer on the other machine); still occasionally getting a ProcessLookupError.

@adqm adqm mentioned this issue Jun 9, 2016
@adqm
Copy link
Member

adqm commented Jun 9, 2016

@ellocogato, @OllieTerrance: I just put in #1192, which for me fixes the issue with xonsh backgrounding itself. It required hooking in to Cygwin's libc implementation via ctypes and it took longer than I would care to admit, because of some stupid assumptions on my part (I assumed some constants would be the same under Cygwin and GNU/Linux, but they aren't...)

but anyway, it's in! Would you mind giving it a look?

I still haven't been able to reproduce the issue with the ProcessLookupError, but that is still on my list. Have either of you had any luck with finding something that consistently reproduces it?

@jivanyatra
Copy link

jivanyatra commented Jun 9, 2016

I'm also attempting to use this on Cygwin (x64). After the 0.3.4 update, I run xonsh, then run pwd, and it gets sent to the background:

Yatri@unnamed ~ $ pwd
/home/Yatri

[1]+  Stopped                 xonsh
150 Yatri@unnamed:~$

fg brings xonsh back to the foreground

When I try to get my xonfig:

150 Yatri@unnamed:~$ python3
Python 3.4.3 (default, May  5 2015, 17:58:45)
[GCC 4.9.2] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from xonsh import xonfig
>>> print(xonfig.main([]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/site-packages/xonsh/xonfig.py", line 468, in main
    return _MAIN_ACTIONS[ns.action](ns)
  File "/usr/lib/python3.4/site-packages/xonsh/xonfig.py", line 339, in _info
    ('shell type', builtins.__xonsh_env__.get('SHELL_TYPE')),
AttributeError: 'module' object has no attribute '__xonsh_env__'
>>>

Not sure what I did wrong. It sort of worked with 0.3.3 (huge delays upwards of half a second to a second after running pwd or ls), but after the update, this happens after every command I've tried.

@adqm
Copy link
Member

adqm commented Jun 9, 2016

Hi @jivanyatra! There is a workaround for the fg issue described here (and that's the issue that #1192 should fix).

As for the other issue...it's hard to say. I think the xonfig stuff really is intended to be run from within xonsh itself, rather than from a separate Python process. So I would say the easiest way to get your xonfig would be to exec xonsh and then run xonfig from there.

@adqm adqm self-assigned this Jun 9, 2016
@jivanyatra
Copy link

jivanyatra commented Jun 9, 2016

Thanks @adqm !! So, exec xonsh now works, and xonfig outputs:

+------------------+----------------+
| xonsh            | 0.3.4          |
| Python           | 3.4.3          |
| PLY              | 3.7            |
| have readline    | True           |
| prompt toolkit   | 1.0.0          |
| shell type       | prompt_toolkit |
| pygments         | 2.1.3          |
| on posix         | True           |
| on linux         | False          |
| on darwin        | False          |
| on windows       | False          |
| on cygwin        | True           |
| is superuser     | False          |
| default encoding | utf-8          |
+------------------+----------------+

I saw that I didn't have prompt_toolkit or pygments installed, so I installed them. Now, I get this result from pwd:

Yatri@unnamed ~ $ pwd
      0 [main] python3 12124 fork: child -1 - CreateProcessW failed for '', errno 2
xonsh: subprocess mode: command not found: pwd
Did you mean one of the following?
    cd:    Alias
    pip:   Command (/usr/bin/pip)
    popd:  Alias
    awk:   Command (/usr/bin/awk)
    mad:   Command (/usr/bin/mad)
2219966 [main] python3 12124 fork: child -1 - CreateProcessW failed for '', errno 2

EDIT: okay so the above no longer occurs. I exited xonsh, ran exec xonsh again, and now ls and cd run correctly, and pwd yields:

Yatri@unnamed ~ $ pwd
/home/Yatri
ProcessLookupError: [Errno 3] No such process

@adqm
Copy link
Member

adqm commented Jun 9, 2016

@jivanyatra, thanks! That ProcessLookupError is the one I haven't yet been able to reproduce on my own machines. But thanks for this information. I can maybe try running with prompt_toolkit to see if that is somehow responsible (I had been using the readline shell), but other than that, our configurations look pretty much the same. Anyway, I'll keep trying that. And #1192 should make it into the next release, so we'll at least be making little steps toward full Cygwin support!

@jivanyatra
Copy link

If there's anything else I can provide, I'm happy to help. I'm a new python dev and this project definitely caught my attention. I'd love to contribute as well, but I'm not sure what I could do with my admittedly novice skill level.

@ellocogato
Copy link

@adqm I tested #1192 and it looks like it properly fixes the issue of xonsh backgrounding itself.

@ellocogato
Copy link

ellocogato commented Jun 10, 2016

@adqm Here's an interesting twist. I still get the occasional ProcessLookupError, and now (with #1192 applied), xonsh will background itself whenever the error occurs.

FYI, if I issue the command while True: pwd, then I will always eventually get the ProcessLookupError

@adqm
Copy link
Member

adqm commented Jun 10, 2016

@ellocogato: Weird! Okay, so maybe this makes it even more crucial to fix that issue. I wish I could reproduce it personally (I still haven't been able to).

Is that backgrounding also fixed with exec xonsh, or is it actually more broken than before?

@ellocogato
Copy link

@adqm OK, one more thing to throw in the mix. Not sure if this was happening before #1192, but I just noticed that sometimes instead of a ProcessLookupError, I get a PermissionError:

Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 154, in default
    run_compiled_code(code, self.ctx, None, 'single')
  File "/usr/lib/python3.4/site-packages/xonsh/codecache.py", line 57, in run_compiled_code
    func(code, glb, loc)
  File "<xonsh-code>", line 1, in <module>
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 607, in subproc_captured_hiddenobject
    return run_subproc(cmds, captured='hiddenobject')
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 509, in run_subproc
    wait_for_active_job()
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 153, in wait_for_active_job
    _continue(active_task)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 74, in _continue
    _send_signal(job, signal.SIGCONT)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 29, in _send_signal
    os.killpg(job['pgrp'], signal)
PermissionError: [Errno 13] Permission denied

@adqm
Copy link
Member

adqm commented Jun 10, 2016

I feel like that had been reported elsewhere as well, and I think I know of a likely fix. I'll try to put in another PR tonight. Do you mind helping to test if I'm not able to reproduce myself?

@ellocogato
Copy link

Regarding exec xonsh, after #1192 everything appears to behave the same as before. No backgrounding during normal operation or when an error occurs.

@adqm
Copy link
Member

adqm commented Jun 10, 2016

Alright, great! Then we've just got that one last thing to track down, it seems, before this works as expected on Cygwin. My hopes are high!

@jivanyatra
Copy link

Okay, after more testing, I'm seeing the same thing as @ellocogato. And there is no more backgrounding occurring.

@adqm
Copy link
Member

adqm commented Jun 10, 2016

Great; thanks for all the help!

@vmiheer
Copy link

vmiheer commented Jun 10, 2016

This was long thread so haven't looked at all of it.
I can run xonsh on cygwin64 python 3.4.
Problem I see is when I type ls command it shows the output and then xonsh goes to background bringing me back to bash.
Doing fg in bash brings me back to xonsh but only till I hit ls again (which is so prevalent).

@Terrance
Copy link
Contributor

This seems to be working much better now. 😄 Only thing I'm seeing now is the 2-3 second delay redrawing the prompt, which was mentioned earlier -- is anyone else still seeing this?

($PROMPT = '> ' removes the delay, so I guess it's the prompt formatting here. It's also a bit slow on my Arch RPi, but I'm assuming the Pi is to blame for that.)

@adqm
Copy link
Member

adqm commented Jun 10, 2016

@vmiheer, yes this is a known issue that should be fixed in the next release (via #1192 and #1193). As a workaround in the meantime, you should be able to use exec xonsh instead of just xonsh to open the shell.

@OllieTerrance: Great! Glad to hear it is working now. We will get this into the next release for sure. It is also true that #1166 (also going in to the next release) will remove the VCS branch information from the prompt, which should speed things up drastically while still giving a somewhat nice looking prompt. As for the RPi, xonsh is pretty heavy-weight for a shell, so I expect there isn't much we can do about it on the pi.

@ellocogato
Copy link

@adqm PR #1193 does prevent the occasional ProcessLookupError and PermissionError.

But now I get some new ones! I have seen a BlockingIOError (just once so far), OSError, and BrokenPipeError. These all occur in os.killpg(). If I handle BlockingIOError and OSError in addition to the others, then that seems to cover it (I guess BrokenPipeError is a subclass of one of the other two).

Also note that I put some debugging output in this error handling code, and I noticed that every time killpg fails, the kill also fails. (killpg can be any of the four errors, but kill is always a ProcessLookupError.) So maybe the kill part is unnecessary? If you want to see this in action: http://pastebin.com/A7NWe6QT

Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 154, in default
    run_compiled_code(code, self.ctx, None, 'single')
  File "/usr/lib/python3.4/site-packages/xonsh/codecache.py", line 57, in run_compiled_code
    func(code, glb, loc)
  File "<xonsh-code>", line 1, in <module>
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 607, in subproc_captured_hiddenobject
    return run_subproc(cmds, captured='hiddenobject')
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 509, in run_subproc
    wait_for_active_job()
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 170, in wait_for_active_job
    _continue(active_task)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 91, in _continue
    _send_signal(job, signal.SIGCONT)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 33, in _send_signal
    os.killpg(job['pgrp'], signal)
BlockingIOError: [Errno 11] Resource temporarily unavailable

[1]+  Stopped                 xonsh
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/xonsh/base_shell.py", line 154, in default
    run_compiled_code(code, self.ctx, None, 'single')
  File "/usr/lib/python3.4/site-packages/xonsh/codecache.py", line 57, in run_compiled_code
    func(code, glb, loc)
  File "<xonsh-code>", line 1, in <module>
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 585, in subproc_captured_stdout
    return run_subproc(cmds, captured='stdout')
  File "/usr/lib/python3.4/site-packages/xonsh/built_ins.py", line 509, in run_subproc
    wait_for_active_job()
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 168, in wait_for_active_job
    _continue(active_task)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 89, in _continue
    _send_signal(job, signal.SIGCONT)
  File "/usr/lib/python3.4/site-packages/xonsh/jobs.py", line 32, in _send_signal
    os.killpg(job['pgrp'], signal)
OSError: [Errno 22] Invalid argument

[1]+  Stopped                 xonsh

@adqm
Copy link
Member

adqm commented Jun 10, 2016

@ellocogato: just updated the patch. do you mind giving it another shot? thanks for all the help with testing this!

@ellocogato
Copy link

@adqm: the update to the patch seems to completely resolve the issue. Is there any consequence of the fact that these errors are occurring (and that they are now being ignoring)?

@adqm
Copy link
Member

adqm commented Jun 10, 2016

@ellocogato: maybe... i guess in the worst case, there could still be a process floating around someplace. i'm not too worried about that, though, since i would imagine the only way we could hit some of these errors is if the processes are already dead and gone...

@adqm
Copy link
Member

adqm commented Jun 13, 2016

Alright, I think we've mostly taken care of these issues. I'm going to close for now. Feel free to re-open (or to open another issue) if problems persist under Cygwin.

@adqm adqm closed this as completed Jun 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

9 participants