Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reloader doesn't add ".exe" to file on Windows #1614

Closed
peppobon opened this issue Jul 17, 2019 · 22 comments
Closed

Reloader doesn't add ".exe" to file on Windows #1614

peppobon opened this issue Jul 17, 2019 · 22 comments
Labels
Milestone

Comments

@peppobon
Copy link

@peppobon peppobon commented Jul 17, 2019

if I run a flask app under 0.15.5 with
FLASK_ENV=development
the dev server throws an error.

With FLASK_ENV=production the server starts ok

After reinstalling 0.15.4 the same app works ok (both in dev and prod mode).

Windows/py3.6.7/pipenv

Kind regards

@ad-65
Copy link

@ad-65 ad-65 commented Jul 17, 2019

Same issue using Flask 1.1.1/Werkzeug 0.15.5 on Windows 10, python 2.7.16, in a virtual environment. Looks like this: pallets/flask#1829 but it was fixed a long time ago.

Loading

@davidism davidism closed this Jul 18, 2019
@davidism davidism changed the title Werkzeug==0.15.5 broken in dev mode ? Reloader doesn't add ".exe" to file on Windows Jul 18, 2019
@davidism davidism reopened this Jul 18, 2019
@davidism davidism added this to the 0.15.6 milestone Jul 18, 2019
@Foknetics
Copy link

@Foknetics Foknetics commented Jul 18, 2019

Noticed this with the release of 0.15.5 as well with an application factory pattern, executing from Windows 10, Python 3.6.2, Flask 1.0.3, via Powershell. Created an example repo here. The change seems to be caused by the function _get_args_for_reloading() in _reloader.py

in 0.15.4 pymodule (line 108) gets set to .flask from the += "." + name which strips down to just flask in rv.

in 0.15.5 pymodule (line 109) gets set to C:\Users\conno\Desktop\werk_ex\venv\Scripts\flask, which strips to the same thing in rv
if os.path.isfile(py_script): (line 100) evaluating to false is really what causes the change from an execution perspective between the two versions.

Loading

@davidism
Copy link
Member

@davidism davidism commented Jul 18, 2019

I can't reproduce this. I have Flask 1.1.1 and Werkzeug 0.15.5, and tried flask run with the reloader in cmd and powershell, and both worked as expected.

Loading

@ad-65
Copy link

@ad-65 ad-65 commented Jul 18, 2019

I tried this again on another Windows 10 machine with a system install of Python 2.7.15 (32-bit) and Python 3.7.2 (32-bit). The following steps (in powershell) reproduce the behavior for me. FLASK_ENV=production works fine, FLASK_ENV=development causes the error.

mkdir C:\Temp\flask_error
cd C:\Temp\flask_error
py -2 -m virtualenv env
.\env\Scripts\activate
pip install flask
echo $null >> hello.py

# Add content to hello.py, see below.

$env:FLASK_APP = "hello.py"
$env:FLASK_ENV = "development"
flask run

 * Serving Flask app "hello.py" (lazy loading)
 * Environment: development
 * Debug mode: on
 * Restarting with stat
c:\temp\flask_error\env\scripts\python.exe: No module named C:\Temp\flask_error\env\Scripts\flask

$env:FLASK_ENV = "production"
flask run

 * Serving Flask app "hello.py"
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

The contents of hello.py are from the flask tutorial:

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

Loading

@davidism
Copy link
Member

@davidism davidism commented Jul 18, 2019

I've been discussing this on Discord too, and no matter what I try, everything works for me. I'm doing the same thing everyone else is to set up an env and run it. I'm going to have a hard time fixing this if I can't observe the behavior to debug it. Someone who has the issue is going to need to debug _get_args_for_reloader() and see what values it's getting along the way and figure out where they're coming from.

Loading

@davidism
Copy link
Member

@davidism davidism commented Jul 18, 2019

If someone can do git bisect to find the responsible commit that would help too.

Loading

@asottile
Copy link
Contributor

@asottile asottile commented Jul 18, 2019

here's the argv / executable for that:

$ FLASK_ENV=development flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
> c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages\werkzeug\_reloader.py(68)_get_args_for_reloading()
-> rv = [sys.executable]
(Pdb) p sys.executable
'c:\\users\\anthony\\appdata\\local\\temp\\x\\venv\\scripts\\python.exe'
(Pdb) p sys.argv
['C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\Scripts\\flask', 'run']

if I use -m flask run I get this:

$ FLASK_ENV=development python -m flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
> c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages\werkzeug\_reloade
r.py(68)_get_args_for_reloading()
-> rv = [sys.executable]
(Pdb) p sys.executable
'C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\Scripts\\python.exe'
(Pdb) p sys.argv
['C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\lib\\site-packages\\flask\\
__main__.py', 'run']
(Pdb) c
 * Debugger is active!
 * Debugger PIN: 132-133-383
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Loading

@asottile
Copy link
Contributor

@asottile asottile commented Jul 18, 2019

oops pressed enter too quickly, I have more info to post

Loading

@davidism
Copy link
Member

@davidism davidism commented Jul 18, 2019

It's expected that flask and -m flask will produce different args. The problem is that for flask you should be getting flask.exe, and the code around that shouldn't have changed.

Loading

@asottile
Copy link
Contributor

@asottile asottile commented Jul 18, 2019

reverting e686ae7 fixes the issue

here's where things appear to go wrong:

$ FLASK_ENV=development flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
> c:\users\anthony\appdata\local\temp\x\werkzeug\src\werkzeug\_reloader.py(95)_get_args_for_reloading()
-> if sys.argv[0] == "-m":
(Pdb) n
> c:\users\anthony\appdata\local\temp\x\werkzeug\src\werkzeug\_reloader.py(101)_get_args_for_reloading()
-> if os.path.isfile(py_script):
(Pdb) py_script
'C:\\Users\\Anthony\\AppData\\Local\\Temp\\x\\venv\\Scripts\\flask'
(Pdb) os.path.isfile(py_script)
False
(Pdb) list
 96                 # Flask works around previous behavior by putting
 97                 # "-m flask" in sys.argv.
 98                 # TODO remove this once Flask no longer misbehaves
 99                 args = sys.argv
100             else:
101  ->             if os.path.isfile(py_script):
102                     # Rewritten by Python from "-m script" to "/path/to/script.py".
103                     py_module = __main__.__package__
104                     name = os.path.splitext(os.path.basename(py_script))[0]
105
106                     if name != "__main__":

Loading

@asottile
Copy link
Contributor

@asottile asottile commented Jul 18, 2019

that doesn't solve the full mystery though -- why are you unable to reproduce this?

I have a hunch, or at least, I've found a way to not reproduce it:

$ pip install flask --no-binary :all:  # previously I uninstalled flask
Collecting flask
  Downloading https://files.pythonhosted.org/packages/2e/80/3726a729de758513fd3dbc64e93098eb009c49305a97c6751de55b20b694/Flask-1.1.1.tar.gz (625kB)
Requirement already satisfied: Werkzeug>=0.15 in c:\users\anthony\appdata\local\temp\x\werkzeug\src (from flask) (1.0.0.dev0)
Requirement already satisfied: Jinja2>=2.10.1 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages (from flask) (2.10.1)
Requirement already satisfied: itsdangerous>=0.24 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages (from flask) (1.1.0)
Requirement already satisfied: click>=5.1 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages (fromflask) (7.0)
Requirement already satisfied: MarkupSafe>=0.23 in c:\users\anthony\appdata\local\temp\x\venv\lib\site-packages(from Jinja2>=2.10.1->flask) (1.1.1)
Installing collected packages: flask
  Running setup.py install for flask: started
    Running setup.py install for flask: finished with status 'done'
Successfully installed flask-1.1.1
(venv)
Anthony@AnthonysDesktop MINGW64 /tmp/x/werkzeug (master)
$ FLASK_ENV=development flask run
 * Environment: development
 * Debug mode: on
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 158-953-008
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Loading

@frostming
Copy link

@frostming frostming commented Jul 18, 2019

Same issue here, on windows 10 Python 3.7.4, Flask 1.1.1/werkzeug 0.15.4

Git bisect shows e686ae7 is the first bad commit.

On line 106:

if os.path.isfile(py_script):
    # Rewritten by Python from "-m script" to "/path/to/script.py".

This condition bypasses on windows as the script path is without .exe, so that py_module remains to be an empty string.

Loading

@davidism
Copy link
Member

@davidism davidism commented Jul 18, 2019

Ah, I was installing Flask by doing git checkout 1.1.1 then pip install -e ., which I guess produces different entry point behavior. I wonder if that should be reported to setuptools or pip.

It's strange that the Windows installed entry point makes it look like a python -m flask invocation (from the perspective of __main__.__package__) but that it doesn't get processed correctly back into python -m flask.

I can't just revert that commit, because it fixes another problem. It seems like we need to figure out some better heuristics for the different run cases.

Loading

@Tinels972
Copy link

@Tinels972 Tinels972 commented Aug 13, 2019

Hello,

I've had a similar issue where I launch a Flask application from an exe in the Scripts folder of the python environment (built using the console_scripts in setuptools).

Basically the reloader thinks this is a regular python package and not an exe because the __package__ in sys.module['__main__'] is an empty string and not None, so I don't enter the condition in _get_args_for_reloading where it adds the .exe in the reloader: if getattr(__main__, "__package__", None) is None:

The workaround to get my application working was to force __package__ to None for now, but this is far from ideal.

Let me know if I'm missing something or if I'm off base in what I just described.

Loading

@samayo
Copy link

@samayo samayo commented Aug 15, 2019

How do we remove werkzeug library from flask? Flask is tagged v1 and we can't even run hello world with it without fatal error

Loading

@ThiefMaster
Copy link
Member

@ThiefMaster ThiefMaster commented Aug 15, 2019

How do we remove werkzeug library from flask? Flask is tagged v1 and we can't even run hello world with it without fatal error

you can't and you don't. why would you want to do that? flask requires werkzeug. Use the latest werkzeug release together with the latest flask release.

Loading

@carc1n0gen
Copy link

@carc1n0gen carc1n0gen commented Aug 16, 2019

@samayo while we're at it, let's remove python from flask too. </sarcasm>

More serious: the workaround is python -m flask run until the next Werkzeug release. Calm down.

Loading

@davidism
Copy link
Member

@davidism davidism commented Aug 16, 2019

If anyone wants to look at this, I'd be very happy. I haven't had much free time lately.

Loading

@frostming
Copy link

@frostming frostming commented Aug 16, 2019

@davidism Working on it, it turned out to be an easy fix after some research.

Loading

@davidism
Copy link
Member

@davidism davidism commented Aug 31, 2019

#1627 does work, but goes down the wrong code path when running an example.py module as python -m example. Given that -m example is probably a very uncommon way to run a single module, and given that the reloader still works, I'm willing to go with this change if we can't quickly find a better solution.

Loading

@davidism
Copy link
Member

@davidism davidism commented Aug 31, 2019

I added some debug statements to werkzeug._reloader._get_args_for_reloading to demonstrate the ways the code paths are triggered on Windows. The first line is sys.executable and sys.argv, the last line is the command passed to subprocess.call.

Show debug output
  • Installed in editable mode (pip install -e .)

    • flask run

      *  c:\users\david\.virtualenvs\flask\scripts\python.exe ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask', 'run']
      * __package__ is None, a file
      * adding .exe: 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask.exe'
      * dropping sys.executable
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask.exe', 'run']
      
    • python -m flask run

      *  C:\Users\David\.virtualenvs\flask\Scripts\python.exe ['c:\\users\\david\\projects\\flask\\src\\flask\\__main__.py', 'run']
      * __package__ is not None, a module: 'flask'
      * script is a file, rewritten -m: 'C:\\Users\\David\\.virtualenvs\\flask\\lib\\site-packages\\flask\\__main__.py'
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\python.exe', '-m', 'flask', 'run']
      
    • python example.py

      *  C:\Users\David\.virtualenvs\flask\Scripts\python.exe ['example.py']
      * __package__ is None, a file
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\python.exe', 'C:\\Users\\David\\Projects\\flask\\example.py']
      
    • python -m example

      *  C:\Users\David\.virtualenvs\flask\Scripts\python.exe ['C:\\Users\\David\\Projects\\flask\\example.py']
      * __package__ is not None, a module: ''
      * script is a file, rewritten -m: 'C:\\Users\\David\\Projects\\flask\\example.py'
      * not __main__: '.example'
      *  ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\python.exe', '-m', 'example']
      
  • Installed from dist (pip install ., dist/flask.tar.gz, dist/flask.whl)

    • flask run (fail)

      *  c:\users\david\.virtualenvs\flask\scripts\python.exe ['C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask', 'run']
      * __package__ is not None, a module: ''
      * module is not a file: 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask'
      *  ['c:\\users\\david\\.virtualenvs\\flask\\scripts\\python.exe', '-m', 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask', 'run']
      
      c:\users\david\.virtualenvs\flask\scripts\python.exe: Error while finding module specification for 'C:\\Users\\David\\.virtualenvs\\flask\\Scripts\\flask' (ModuleNotFoundError: No module named 'C:\\Users\\David\\')
      
    • Other methods worked the same as -e

Ultimately, this is probably a bug with setuptools, since the entry point it creates on Windows behaves differently for editable vs dist installs.

Loading

@davidism
Copy link
Member

@davidism davidism commented Aug 31, 2019

Closed in #1627

Loading

@davidism davidism closed this Aug 31, 2019
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants