runserver_plus: watchdog >=2.3.* causes unwanted behavior

[srtab]

Hi,

We are using runserver_plus in our projects, and yesterday, we started to notice a problem on change detection. Basically, watchdog started to detect changes only by loading a page on the browser. In our investigation, we discovered that watchdog added a new feature to detect FileOpenedEvent which appears to trigger change detection without any actual file changes. We suspect that this new feature is causing the issue.
Does this make any sense?

[YAmikep]

Not sure if it's related but I am seeing some weird behavior too.
Every time I load a page, it seems like some changes are detected even if none of my code source has changed.
Also, it's constantly detecting changes and restarting the server while nothing is being edited, which makes it unusable.
What is weird is that the detected files are libraries files like django files.

Does anyone have this issue?
I am using WSL and pyenv if it makes a difference.

127.0.0.1 - - [05/Mar/2023 02:58:31] "GET /admin/ HTTP/1.1" 200 -
 * Restarting with watchdog (inotify)

Django version 4.2b1, using settings 'foo.settings.dev'
Development server is running at http://[127.0.0.1]:51180/
Using the Werkzeug debugger (http://werkzeug.pocoo.org/)
Quit the server with CONTROL-C.
 * Debugger is active!
 * Debugger PIN: 417-317-591
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/contrib/sessions/backends/db.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/contrib/sessions/backends/base.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/contrib/auth/__init__.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/contrib/auth/middleware.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/contrib/admin/sites.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/contrib/staticfiles/handlers.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/werkzeug/debug/__init__.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/werkzeug/serving.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/3.11.2/lib/python3.11/http/server.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/contrib/auth/backends.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/conf/__init__.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/utils/functional.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/views/debug.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/debug_toolbar/panels/settings.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/debug_toolbar/middleware.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/core/handlers/exception.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/core/handlers/base.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/venv/lib/python3.11/site-packages/django/core/handlers/wsgi.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/3.11.2/lib/python3.11/socketserver.py', reloading
 * Detected change in '/home/michael/.pyenv/versions/3.11.2/lib/python3.11/threading.py', reloading

[srtab]

Hi @YAmikep,

We have the same issue here.
The files where the changes are detected are the files imported on the executed Python when reloading a page. That's why I suspect that the problem comes from the new feature added to watchdog.

[YAmikep]

Ha right, if I pin watchdog to 2.2, I don't see the issue anymore. It might be worth filling this bug on the watchdog project then maybe?

[srtab]

I don't think that's a watchdog bug. What I suspect is that django-extensions is catching all type of events triggered by watchdog, including FileOpenedEvent.

I will try to contribute with a pull request in the next days.

[YAmikep]

I see what you mean. Makes sense now. 👍

[srtab]

After some digging, I found out that this problem is not directly related with django-extensions, but with werkzeug.

It seems that the problem has been solved on main branch, here's the related issue: pallets/werkzeug#2603.

[michal-stachura]

Ha right, if I pin watchdog to 2.2, I don't see the issue anymore. It might be worth filling this bug on the watchdog project then maybe?

I have changed Werkzug to 2.2.3 and still have this issue on cookiecutter Django with Docker. local requirements below. For me only works if I start server with exec python manage.py runserver 0.0.0.0:8000 instead of exec python manage.py runserver_plus 0.0.0.0:8000

--- local requirements ---

-r base.txt

Werkzeug[watchdog]==2.2.3 # https://github.com/pallets/werkzeug
ipdb==0.13.11  # https://github.com/gotcha/ipdb
psycopg2==2.9.5  # https://github.com/psycopg/psycopg2
watchfiles==0.18.1  # https://github.com/samuelcolvin/watchfiles

# Testing
# ------------------------------------------------------------------------------
mypy==0.982  # https://github.com/python/mypy
django-stubs==1.13.1  # https://github.com/typeddjango/django-stubs
pytest==7.2.0  # https://github.com/pytest-dev/pytest
pytest-sugar==0.9.6  # https://github.com/Frozenball/pytest-sugar
djangorestframework-stubs==1.8.0  # https://github.com/typeddjango/djangorestframework-stubs

# Documentation
# ------------------------------------------------------------------------------
sphinx==5.3.0  # https://github.com/sphinx-doc/sphinx
sphinx-autobuild==2021.3.14 # https://github.com/GaretJax/sphinx-autobuild

# Code quality
# ------------------------------------------------------------------------------
flake8==6.0.0  # https://github.com/PyCQA/flake8
flake8-isort==6.0.0  # https://github.com/gforcada/flake8-isort
coverage==7.0.4  # https://github.com/nedbat/coveragepy
black==22.12.0  # https://github.com/psf/black
pylint-django==2.5.3  # https://github.com/PyCQA/pylint-django
pylint-celery==0.3  # https://github.com/PyCQA/pylint-celery
pre-commit==2.21.0  # https://github.com/pre-commit/pre-commit

# Django
# ------------------------------------------------------------------------------
factory-boy==3.2.1  # https://github.com/FactoryBoy/factory_boy

django-debug-toolbar==3.8.1  # https://github.com/jazzband/django-debug-toolbar
django-extensions==3.2.1  # https://github.com/django-extensions/django-extensions
django-coverage-plugin==3.0.0  # https://github.com/nedbat/django_coverage_plugin
pytest-django==4.5.2  # https://github.com/pytest-dev/pytest-django

[YAmikep]

@michal-stachura
Try to pin watchdog too: watchdog==2.2