Python Shell Command Fails on Windows

[ghferrari]

I'm using emacs-w64 on Windows 7 (https://sourceforge.net/projects/emacsbinw64/). When I run

M-x run-python

I get the following error trace:

Debugger entered--Lisp error: (file-error "Searching for program" "no such file or directory" "c\\:/Users/Geoff\\") start-process("Python" #<buffer *Python*> "c\\:/Users/Geoff\\" "Ferrari/AppData/Local/Programs/Python/Python35-32/python.exe" "-i")

The problem can be traced to two things.

1. My Windows username has a space in it.
2. I've set my shell-file-name to use zsh.exe from cygwin.

If I use zsh.exe as my shell-file-name then the following code produces this result:

(shell-quote-argument "c:/Users/Geoff Ferrari/AppData/Local/Programs/Python/Python35-32/python.exe")
"c\\:/Users/Geoff\\ Ferrari/AppData/Local/Programs/Python/Python35-32/python.exe -i"'

However, if I leave my shell-file-name as the default cmdproxy.exe supplied by emacs, then it produces this result:
"\"c:/Users/Geoff Ferrari/AppData/Local/Programs/Python/Python35-32/python.exe\""

As you can see, using zsh.exe as my shell-file-name produces different results. In particular, it adds \ before the space in my username.

These differences mean that python-shell-make-comint fails for me because split-string-and-unquote splits my python-shell-command on the space in my username. The result that you see in the error message above is that the command given to start-process is just the path up to the space in my username c:/Users/Geoff", which of course is not an existing command, while the rest of the path is passed as the first of the two arguments: "Ferrari/AppData/Local/Programs/Python/Python35-32/python.exe" "-i"

I don't know whether it's python.el that's at fault here, but I thought I'd make you aware of this first. It would be good to have a proper resolution to the issue. But for the moment I'm working around the problem like this:

Before changing shell-file-name I store its original value like so:
(defvar default-shell-file-name shell-file-name)

Then I use the following advice on python-shell-parse-command:

        (defun python-shell-parse-command--adjust-shell-command (orig-fun &rest args)
            (let ((shell-file-name default-shell-file-name))
                (apply orig-fun args)))

        (advice-add 'python-shell-parse-command :around #'python-shell-parse-command--adjust-shell-command)

I'm no expert here, but I guess that the problem is that start-process is going to use cmdproxy.exe even if I've set my shell-file-name to zsh.exe . So that command that is passed to start-process still needs to be quoted for Windows rather than for zsh.exe.

[rthoma]

Amazing. I came here to post this exact same issue. In my case I'm launching Emacs from git bash on Windows, but I'm seeing the same behavior. When launching Emacs from git bash python-shell-parse-command returns "c\\:/Anaconda2/Scripts/ipython.exe -i ", which doesn't work. When launching Emacs from the cmd shell, python-shell-parse-command returns "\"c:/Anaconda2/Scripts/ipython.exe\" -i" which works just fine. I'll try the workaround suggested by @ghferrari for now.

Edit 1: Weirdly, if I just take the code inside python-shell-parse-command and evaluate it in the minibuffer, it returns "c:/Anaconda2/Scripts/ipython.exe -i". Any clue what might cause this discrepancy?

Edit 2: If I open python.el after Emacs has loaded and evaluate the buffer, python-shell-parse-command returns the correct string. It looks like either process-environment or exec-path is changing after Emacs loads, which causes executable-find to return a different string.

Edit 3: So here's what happened. I updated the version of python-shell-parse-command in python.el by pulling an updated function from the version in the github master branch. I then failed to byte-compile the file, which was causing the discrepancy mentioned above. Emacs was internally relying on the byte-compiled version, while I was running the updated version from the uncompiled .el file. See my note below for a possible solution.

[ghferrari]

@rthoma Thanks for adding to my bug report. It seems the problem is not just caused by having a space in your Windows username.

[rthoma]

@ghferrari It appears this function has been fixed in the github repo, but not in the version that comes with Emacs 24.5.1. I was able to resolve my issue by opening my python.el and replacing the python-shell-parse-command function with the version from the github repo. Reposted here for convenience:

(defun python-shell-parse-command ()
  "Calculate the string used to execute the inferior Python process."
  (let ((process-environment (python-shell-calculate-process-environment))
        (exec-path (python-shell-calculate-exec-path)))
  (format "%s %s"
          (executable-find python-shell-interpreter)
          python-shell-interpreter-args)))

Then use M-x to run byte-compile-file. This worked for me, so hopefully it will address your problem too.

Note/Caution: I tried to clone the github repo and byte compile the version of python.el in the master branch. This did NOT work. I don't recommend it. So I reverted to the version of python.el that shipped with Emacs 24.5.1 with the modified python-shell-parse-command function. This seemed to keep everything in working order.

Another thing to try if the above does not work is to use the version of python-shell-parse-command that ships with Emacs 25.1, given below. Note that I personally have not tried this because the option above solved my problem.

(defun python-shell-parse-command ()
  "Calculate the string used to execute the inferior Python process."
  (format "%s %s"
          (shell-quote-argument python-shell-interpreter)
          python-shell-interpreter-args))

PS - This repo looks dead. The official python.el is now maintained along with Emacs itself. So I'm not sure where python.el bugs are getting reported now.

[ghferrari]

Thanks, rthoma but your solution doesn't work for me. I just get the error
Spawning child process: invalid argument . I think this has to do with the unquoted python shell path, similar to the above.

[rthoma]

Hmmm... What does (executable-find python-shell-interpreter) return for you using M-:? For me it yields "c:/Anaconda2/Scripts/ipython.exe". So you would want (executable-find python-shell-interpreter) to yield "c:/Users/Geoff Ferrari/AppData/Local/Programs/Python/Python35-32/python.exe". If (executable-find python-shell-interpreter) returned that string and you were using the first option in my previous post, Python should start up just fine. You might want to ignore the second option. That seems dependent on some changes made in Emacs 25.

[ghferrari]

Hi @rthoma, yes, (executable-find python-shell-interpreter) has the value you guessed, "c:/Users/Geoff Ferrari/AppData/Local/Programs/Python/Python35-32/python.exe", but the function run-python fails with this value on the line

(python-shell-make-comint cmd (python-shell-get-process-name dedicated) show)

The version of python-shell-parse-command from Emacs 25 works fine, however.

Thanks for your help.

[patfla]

Hi - I'm having what's in effect the same problem. Latest version of emacs, 25.1.2; ipython runs for run-python but while producing an Input prompt, no Output.

Was wondering what progress may have been made on the problem?

[rthoma]

Hi, @patfla. It looks like python.el development is no longer being hosted on github. I think it has been brought into the main Emacs development tree. So you won't receive any help from the developers here, more than likely. From your comment, I'm not sure if you're experiencing the same error as @ghferrari and I were. The problem cropped up for us because we are launching Emacs from alternative shells, in his case zsh and in my case git bash. For me, this was causing a fatal error upon M-x run-python. Ipython never loaded on account of python.el producing an incorrect shell command. It sounds like your problem might be unrelated. If you think it is the same problem, then I recommend manually editing python.el by commenting out python-shell-parse-command and replacing it with 1 of the 2 versions reproduced above. Good luck-

[patfla]

You're right @rthoma - I posted here about a problem that I'm having which I saw it reflected in a different github discussion thread. Sort of, since both github tickets are related to problems in running ipython in an windows emacs run-python buffer.

[skangas]

Hi!

This repository is no longer maintained, since python-mode.el is now distributed with Emacs itself. I suggest that you report any issues using the Emacs command:

M-x report-emacs-bug