Unstable (non-idempotent) isort behavior.

[apollo13]

I am seeing something weird on isort 5.6.0 and didn't seem to find something similar in the issue. When running isort tests src against my project (https://github.com/rocketDuck/dobby) multiple times I get the following output:

(.venv) ➜  dobby git:(master) ✗ isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py
(.venv) ➜  dobby git:(master) ✗ isort tests src
(.venv) ➜  dobby git:(master) ✗ isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py

As you can see isort fixes the file multiple times (adding and removing a blank line between imports) without any changes by me. I literally ran those commands one after another.

Additional information:

python --version
Python 3.8.6

Running with --verbose shows that isort sometimes considers dobby FIRST and sometimes THIRDPARTY:

from-type place_module for pathlib returned STDLIB
else-type place_module for pytest returned THIRDPARTY
from-type place_module for dobby.templates returned THIRDPARTY
else-type place_module for click returned THIRDPARTY
else-type place_module for inspect returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for functools returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for httpx returned THIRDPARTY
from-type place_module for jinja2.exceptions returned THIRDPARTY
from-type place_module for .config returned LOCALFOLDER
else-type place_module for os returned STDLIB
else-type place_module for sys returned STDLIB
else-type place_module for time returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER
else-type place_module for functools returned STDLIB
else-type place_module for json returned STDLIB
else-type place_module for os returned STDLIB
else-type place_module for pathlib returned STDLIB
else-type place_module for re returned STDLIB
else-type place_module for yaml returned THIRDPARTY
from-type place_module for dotenv.main returned THIRDPARTY
from-type place_module for jinja2 returned THIRDPARTY
from-type place_module for jinja2.runtime returned THIRDPARTY
from-type place_module for dobby.cli returned THIRDPARTY
else-type place_module for ssl returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for pathlib returned STDLIB
from-type place_module for urllib.parse returned STDLIB
else-type place_module for click returned THIRDPARTY
else-type place_module for httpx returned THIRDPARTY
from-type place_module for httpx._config returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER

vs

from-type place_module for pathlib returned STDLIB
else-type place_module for pytest returned THIRDPARTY
from-type place_module for dobby.templates returned FIRSTPARTY
Fixing /home/florian/dev/dobby/tests/test_templating.py
else-type place_module for click returned THIRDPARTY
else-type place_module for inspect returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for functools returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for httpx returned THIRDPARTY
from-type place_module for jinja2.exceptions returned THIRDPARTY
from-type place_module for .config returned LOCALFOLDER
else-type place_module for os returned STDLIB
else-type place_module for sys returned STDLIB
else-type place_module for time returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER
else-type place_module for functools returned STDLIB
else-type place_module for json returned STDLIB
else-type place_module for os returned STDLIB
else-type place_module for pathlib returned STDLIB
else-type place_module for re returned STDLIB
else-type place_module for yaml returned THIRDPARTY
from-type place_module for dotenv.main returned THIRDPARTY
from-type place_module for jinja2 returned THIRDPARTY
from-type place_module for jinja2.runtime returned THIRDPARTY
from-type place_module for dobby.cli returned FIRSTPARTY
else-type place_module for ssl returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for pathlib returned STDLIB
from-type place_module for urllib.parse returned STDLIB
else-type place_module for click returned THIRDPARTY
else-type place_module for httpx returned THIRDPARTY
from-type place_module for httpx._config returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER

As to why isort behaves differently on subsequent runs I have no idea.

[apollo13]

Isort itself is installed in the current active environment. I am using poetry to manage dependencies. If you clone my project this should be able to reproduce it:

poetry install
poetry shell
isort tests src
isort tests src
isort tests src

[apollo13]

I left my repository at the current broken state. I know that I can use known_first_party and similar config options, but it would be nice if isort were stable either way.

[timothycrosley]

Thank you for reporting! This issue has been resolved in a hotfix release (5.6.1) and your project has been added to isort's integration test suite.

Any time non-idempotent behaviour is observed with isort it is of major concern for the project. I was able to pinpoint the cause of the issue to two things:

1. The just released support for automatic detection of namespace packages misidentifying for repos with no python files in root. This caused the mis-categorization.
2. The use of frozenset (which has non-deterministic ordering) internally to de-dup source paths. This caused the non-idempotent behaviour, and has been an undiscovered bug for some time.

Both of these issues have been resolved.

Thanks!

~Timothy

[apollo13]

Thank you for the quick turnaround and explanation!