Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reloader doesn't add ".exe" to file on Windows #1614

Closed
peppobon opened this issue Jul 17, 2019 · 22 comments

Comments

@peppobon
Copy link

commented Jul 17, 2019

if I run a flask app under 0.15.5 with
FLASK_ENV=development
the dev server throws an error.

With FLASK_ENV=production the server starts ok

After reinstalling 0.15.4 the same app works ok (both in dev and prod mode).

Windows/py3.6.7/pipenv

Kind regards

@ad-65

This comment has been minimized.

Copy link

commented Jul 17, 2019

Same issue using Flask 1.1.1/Werkzeug 0.15.5 on Windows 10, python 2.7.16, in a virtual environment. Looks like this: pallets/flask#1829 but it was fixed a long time ago.

@davidism davidism closed this Jul 18, 2019

@davidism davidism changed the title Werkzeug==0.15.5 broken in dev mode ? Reloader doesn't add ".exe" to file on Windows Jul 18, 2019

@davidism davidism reopened this Jul 18, 2019

@davidism davidism added this to the 0.15.6 milestone Jul 18, 2019

@Foknetics

This comment has been minimized.

Copy link

commented Jul 18, 2019

Noticed this with the release of 0.15.5 as well with an application factory pattern, executing from Windows 10, Python 3.6.2, Flask 1.0.3, via Powershell. Created an example repo here. The change seems to be caused by the function _get_args_for_reloading() in _reloader.py

in 0.15.4 pymodule (line 108) gets set to .flask from the += "." + name which strips down to just flask in rv.

in 0.15.5 pymodule (line 109) gets set to C:\Users\conno\Desktop\werk_ex\venv\Scripts\flask, which strips to the same thing in rv
if os.path.isfile(py_script): (line 100) evaluating to false is really what causes the change from an execution perspective between the two versions.

@davidism

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

I can't reproduce this. I have Flask 1.1.1 and Werkzeug 0.15.5, and tried flask run with the reloader in cmd and powershell, and both worked as expected.

@davidism davidism added the reloader label Jul 18, 2019

@ad-65

This comment has been minimized.

Copy link

commented Jul 18, 2019

I tried this again on another Windows 10 machine with a system install of Python 2.7.15 (32-bit) and Python 3.7.2 (32-bit). The following steps (in powershell) reproduce the behavior for me. FLASK_ENV=production works fine, FLASK_ENV=development causes the error.

mkdir C:\Temp\flask_error
cd C:\Temp\flask_error
py -2 -m virtualenv env
.\env\Scripts\activate
pip install flask
echo $null >> hello.py

# Add content to hello.py, see below.

$env:FLASK_APP = "hello.py"
$env:FLASK_ENV = "development"
flask run

 * Serving Flask app "hello.py" (lazy loading)
 * Environment: development
 * Debug mode: on
 * Restarting with stat
c:\temp\flask_error\env\scripts\python.exe: No module named C:\Temp\flask_error\env\Scripts\flask

$env:FLASK_ENV = "production"
flask run

 * Serving Flask app "hello.py"
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

The contents of hello.py are from the flask tutorial:

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'
@davidism

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

I've been discussing this on Discord too, and no matter what I try, everything works for me. I'm doing the same thing everyone else is to set up an env and run it. I'm going to have a hard time fixing this if I can't observe the behavior to debug it. Someone who has the issue is going to need to debug _get_args_for_reloader() and see what values it's getting along the way and figure out where they're coming from.

@davidism

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

If someone can do git bisect to find the responsible commit that would help too.

@asottile

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

here's the argv / executable for that:

$ FLASK_ENV=development flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
> c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages\werkzeug\_reloader.py(68)_get_args_for_reloading()
-> rv = [sys.executable]
(Pdb) p sys.executable
'c:\\users\\anthony\\appdata\\local\\temp\\x\\venv\\scripts\\python.exe'
(Pdb) p sys.argv
['C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\Scripts\\flask', 'run']

if I use -m flask run I get this:

$ FLASK_ENV=development python -m flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
> c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages\werkzeug\_reloade
r.py(68)_get_args_for_reloading()
-> rv = [sys.executable]
(Pdb) p sys.executable
'C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\Scripts\\python.exe'
(Pdb) p sys.argv
['C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\lib\\site-packages\\flask\\
__main__.py', 'run']
(Pdb) c
 * Debugger is active!
 * Debugger PIN: 132-133-383
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
@asottile

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

oops pressed enter too quickly, I have more info to post

@davidism

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

It's expected that flask and -m flask will produce different args. The problem is that for flask you should be getting flask.exe, and the code around that shouldn't have changed.

@asottile

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

reverting e686ae7 fixes the issue

here's where things appear to go wrong:

$ FLASK_ENV=development flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
> c:\users\anthony\appdata\local\temp\x\werkzeug\src\werkzeug\_reloader.py(95)_get_args_for_reloading()
-> if sys.argv[0] == "-m":
(Pdb) n
> c:\users\anthony\appdata\local\temp\x\werkzeug\src\werkzeug\_reloader.py(101)_get_args_for_reloading()
-> if os.path.isfile(py_script):
(Pdb) py_script
'C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\Scripts\\flask'
(Pdb) os.path.isfile(py_script)
False
(Pdb) list
 96                 # Flask works around previous behavior by putting
 97                 # "-m flask" in sys.argv.
 98                 # TODO remove this once Flask no longer misbehaves
 99                 args = sys.argv
100             else:
101  ->             if os.path.isfile(py_script):
102                     # Rewritten by Python from "-m script" to "/path/to/script.py".
103                     py_module = __main__.__package__
104                     name = os.path.splitext(os.path.basename(py_script))[0]
105
106                     if name != "__main__":
@asottile

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

that doesn't solve the full mystery though -- why are you unable to reproduce this?

I have a hunch, or at least, I've found a way to not reproduce it:

$ pip install flask --no-binary :all:  # previously I uninstalled flask
Collecting flask
  Downloading https://files.pythonhosted.org/packages/2e/80/3726a729de758513fd3dbc64e93098eb009c49305a97c6751de55b20b694/Flask-1.1.1.tar.gz (625kB)
Requirement already satisfied: Werkzeug>=0.15 in c:\users\anthony\appdata\local\temp\x\werkzeug\src (from flask) (1.0.0.dev0)
Requirement already satisfied: Jinja2>=2.10.1 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages (from flask) (2.10.1)
Requirement already satisfied: itsdangerous>=0.24 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages (from flask) (1.1.0)
Requirement already satisfied: click>=5.1 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages (fromflask) (7.0)
Requirement already satisfied: MarkupSafe>=0.23 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages(from Jinja2>=2.10.1->flask) (1.1.1)
Installing collected packages: flask
  Running setup.py install for flask: started
    Running setup.py install for flask: finished with status 'done'
Successfully installed flask-1.1.1
(venv)
Anthony@AnthonysDesktop MINGW64 /tmp/x/werkzeug (master)
$ FLASK_ENV=development flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 158-953-008
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
@frostming

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

Same issue here, on windows 10 Python 3.7.4, Flask 1.1.1/werkzeug 0.15.4

Git bisect shows e686ae7 is the first bad commit.

On line 106:

if os.path.isfile(py_script):
    # Rewritten by Python from "-m script" to "/path/to/script.py".

This condition bypasses on windows as the script path is without .exe, so that py_module remains to be an empty string.

@davidism

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

Ah, I was installing Flask by doing git checkout 1.1.1 then pip install -e ., which I guess produces different entry point behavior. I wonder if that should be reported to setuptools or pip.

It's strange that the Windows installed entry point makes it look like a python -m flask invocation (from the perspective of __main__.__package__) but that it doesn't get processed correctly back into python -m flask.

I can't just revert that commit, because it fixes another problem. It seems like we need to figure out some better heuristics for the different run cases.

@Tinels972

This comment has been minimized.

Copy link

commented Aug 13, 2019

Hello,

I've had a similar issue where I launch a Flask application from an exe in the Scripts folder of the python environment (built using the console_scripts in setuptools).

Basically the reloader thinks this is a regular python package and not an exe because the __package__ in sys.module['__main__'] is an empty string and not None, so I don't enter the condition in _get_args_for_reloading where it adds the .exe in the reloader: if getattr(__main__, "__package__", None) is None:

The workaround to get my application working was to force __package__ to None for now, but this is far from ideal.

Let me know if I'm missing something or if I'm off base in what I just described.

@samayo

This comment has been minimized.

Copy link

commented Aug 15, 2019

How do we remove werkzeug library from flask? Flask is tagged v1 and we can't even run hello world with it without fatal error

@ThiefMaster

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

How do we remove werkzeug library from flask? Flask is tagged v1 and we can't even run hello world with it without fatal error

you can't and you don't. why would you want to do that? flask requires werkzeug. Use the latest werkzeug release together with the latest flask release.

@carc1n0gen

This comment has been minimized.

Copy link

commented Aug 16, 2019

@samayo while we're at it, let's remove python from flask too. </sarcasm>

More serious: the workaround is python -m flask run until the next Werkzeug release. Calm down.

@davidism

This comment has been minimized.

Copy link
Member

commented Aug 16, 2019

If anyone wants to look at this, I'd be very happy. I haven't had much free time lately.

@frostming

This comment has been minimized.

Copy link
Contributor

commented Aug 16, 2019

@davidism Working on it, it turned out to be an easy fix after some research.

@davidism

This comment has been minimized.

Copy link
Member

commented Aug 31, 2019

#1627 does work, but goes down the wrong code path when running an example.py module as python -m example. Given that -m example is probably a very uncommon way to run a single module, and given that the reloader still works, I'm willing to go with this change if we can't quickly find a better solution.

@davidism

This comment has been minimized.

Copy link
Member

commented Aug 31, 2019

I added some debug statements to werkzeug._reloader._get_args_for_reloading to demonstrate the ways the code paths are triggered on Windows. The first line is sys.executable and sys.argv, the last line is the command passed to subprocess.call.

Show debug output
  • Installed in editable mode (pip install -e .)

    • flask run

      *  c:\users\david\.virtualenvs\flask\scripts\python.exe ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask', 'run']
      * __package__ is None, a file
      * adding .exe: 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask.exe'
      * dropping sys.executable
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask.exe', 'run']
      
    • python -m flask run

      *  C:\Users\David\.virtualenvs\flask\Scripts\python.exe ['c:\\users\\david\\projects\\flask\\src\\flask\\__main__.py', 'run']
      * __package__ is not None, a module: 'flask'
      * script is a file, rewritten -m: 'C:\\Users\\David\\.virtualenvs\\flask\\lib\\site-packages\\flask\\__main__.py'
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\python.exe', '-m', 'flask', 'run']
      
    • python example.py

      *  C:\Users\David\.virtualenvs\flask\Scripts\python.exe ['example.py']
      * __package__ is None, a file
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\python.exe', 'C:\\Users\\David\\Projects\\flask\\example.py']
      
    • python -m example

      *  C:\Users\David\.virtualenvs\flask\Scripts\python.exe ['C:\\Users\\David\\Projects\\flask\\example.py']
      * __package__ is not None, a module: ''
      * script is a file, rewritten -m: 'C:\\Users\\David\\Projects\\flask\\example.py'
      * not __main__: '.example'
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\python.exe', '-m', 'example']
      
  • Installed from dist (pip install ., dist/flask.tar.gz, dist/flask.whl)

    • flask run (fail)

      *  c:\users\david\.virtualenvs\flask\scripts\python.exe ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask', 'run']
      * __package__ is not None, a module: ''
      * module is not a file: 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask'
      *  ['c:\\users\\david\\.virtualenvs\\flask\\scripts\\python.exe', '-m', 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask', 'run']
      
      c:\users\david\.virtualenvs\flask\scripts\python.exe: Error while finding module specification for 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask' (ModuleNotFoundError: No module named 'C:\\Users\\David\\')
      
    • Other methods worked the same as -e

Ultimately, this is probably a bug with setuptools, since the entry point it creates on Windows behaves differently for editable vs dist installs.

@davidism

This comment has been minimized.

Copy link
Member

commented Aug 31, 2019

Closed in #1627

@davidism davidism closed this Aug 31, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.