Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reloader option to exclude watch paths #1333

Closed
rotemdan opened this issue Jul 11, 2018 · 7 comments · Fixed by #1861
Closed

Reloader option to exclude watch paths #1333

rotemdan opened this issue Jul 11, 2018 · 7 comments · Fixed by #1861
Labels
Milestone

Comments

@rotemdan
Copy link

rotemdan commented Jul 11, 2018

Seems like the server gets reloaded when gen_py caching files (created by the win32com module) are written or modified (in a temporary directory):

 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\__pycache__\\SpVoice.cpython-36.pyc', reloading
 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\__pycache__\\_ISpeechVoiceEvents.cpython-36.pyc', reloading

This seems to happen quite often. Any way for this to be prevented?

(Using Python 3.6, Flask 1.0.2, Werkzeug 0.14.1 on Win10 64bit. FLASK_ENV=debug, FLASK_DEBUG=1)

Edit: this also causes occasional cache corruptions (causing runtime errors and requiring wiping the gen_py directory to resolve) since it abruptly cuts out the process while it is writing (usually several files) to disk.

@angadkalra
Copy link

set this environment variable: PYTHONDONTWRITEBYTECODE=1

@rotemdan
Copy link
Author

Thanks for the advice. I wiped the temporary gen_py directory and I tried it. Seems like .pyc files aren't written, however there are still .py files being written there. Causing the server to reload:

 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\ISpeechObjectTokens.py', reloading
 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\SpObjectToken.py', reloading
 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\ISpeechObjectToken.py', reloading
 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\SpMemoryStream.py', reloading
 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\ISpeechMemoryStream.py', reloading
 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\SpAudioFormat.py', reloading
 * Detected change in 'C:\\Users\\X\\AppData\\Local\\Temp\\gen_py\\3.6\\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4\\ISpeechAudioFormat.py', reloading

@angadkalra
Copy link

Does your app successfully reload after seeing those lines above?

@rotemdan
Copy link
Author

rotemdan commented Jul 19, 2018

@angadkalra The app reloads (although there is no practical need for it to). Though since the win32com module is writing multiple files to disk and is abruptly terminated by the reloader, it occasionally results in a cache corruption (some files successfully written, some don't or only partially). Next time it loads it may sometimes crash the server with a runtime error of the sort Can't load COM module ... since the cache has been corrupted and only wiping out the gen_py dir would seem to resolve it.

The goal is not to have the server reload when files (including .py) are written to cache or temporary directories. It seems reasonable there's already a way to do it but I'm not sure how to achieve it.

@davidism
Copy link
Member

I realized this will break the reloader if the virtualenv itself is in the temp folder. None of the tools I commonly use do that by default (pew temp envs, tox, nox), and I don't think Pipenv or Poetry do, but there are a lot of other tools and workflows out there.

Instead, let's allow passing a list of exclude_paths like the extra_files. I'm not sure of the exact semantics for this. I know there's a difference between the stat and watchdog reloaders about watching paths. I'm not sure what order extra and exclude should be considered in.

@sphakka
Copy link

sphakka commented Jun 25, 2020

Please consider using wildcards for exclude_paths: I see reload occurring on changing editor's (Emacs) backup files (.#file.py), which is utterly annoying as those are modified automatically without saving the original file.

So, exclude_paths = [ '.#*.py', ... ] would be great, and should possibly be the default with other tricky patterns.

@danihodovic
Copy link

danihodovic commented Sep 3, 2020

#1861

The reloader fails to detect changes if there is a file owned by root and permissions set to 0700 in the project directory. This is common if using docker-compose for development with locally stored database files.

It seems like you can pass ignore_patterns to watchdog. Maybe that's helpful. https://pythonhosted.org/watchdog/api.html#watchdog.events.PatternMatchingEventHandler.ignore_patterns

@davidism davidism added this to the 2.0.0 milestone Oct 28, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
5 participants