Make isort venv, or other environment, independent

[timothycrosley]

This ticket represents the work to make isort work the same, by default independent to what environments are loaded or installed. This is increasingly important as isort is being utilized widely within CI/CD systems, where the likely hood that a build system will have a different venv than a developers machine becomes very high.

Suggested approach:

Default finding should do the following on a per import basis:

• CUSTOM: import name exists in any custom specified import sections.
• FUTURE: import starts with from __future__
• STDLIB: import name exists in comprehensive built in definition of stdlib imports
• FIRSTPARTY: import name starts with .
• FIRSTPARTY: import code exists within the files isort is told to sort.
• FIRSTPARTY: import code exists in same directory as any files isort is told to sort.
• THIRDPARTY: Everything else.

In this way, by default isort will not use any magic and will be fully repeatable 🤞. The older approach should be made available via a flag, such as --environment-identifcation-magic and corresponding config option. No optional dependencies (such as toml, setup.py requirements etc) should be used by default, as using these optional dependencies by default is a recipe for non-repeatable runs and user confusion.

Related:
#1058
#1071
#1104
#1098
#1048
#1007
#952
#910
And more that have been closed from previous magic improvement work, though this is the first magic removal attempt.

[ndevenish]

I just discovered this testing the master branch on our code sorted with 4.3.21 with everything mixing into first-party (within isolated pre-commit hooks is out direct use case here). Good to see it's known.

The change to third-by-default sounds sane; for our use case I'd be happy with that and explicitly specifying first-party, though appreciate some level of heuristic magic might make it more flexible.

Doing so based on where it's run sounds to me like maybe a risk of getting different behaviour based on where it's run from (e.g. along with pre-commit we have weekly jobs that run inside and over whole multi-project trees, in case people haven't been installing the hooks....). Perhaps "FIRSTPARTY is anything above in the current package"?, though I imagine namespace packages make this hard to determine heuristically also.

From these rules It looks like local/relative (.) imports are proposed to be counted as FIRSTPARTY - these are currently LOCAL, I believe? Planning to remove this distinction, or just not specifying here?

[asottile]

fwiw, you may find some prior art in https://github.com/asottile/aspy.refactor_imports (which powers reorder-python-imports) -- notably you'll likely run into the issue of src layouts where you'll want some sort of argument to indicate where the source roots are -- the tool above uses --application-directories

as a workaround, and what's been suggested for usage with pre-commit at least, is to utilize seed-isort-config which leverages aspy.refactor_imports to populate known_third_party and then isort "just works"

I suggested something similar way back in #184 ;)

[beneisner]

Is there any interest in/plan for allowing the [tool.poetry.dependencies] and [tool.poetry.dev-dependencies] to seed the isort THIRDPARTY list?

[saroad2]

When does this fix is about to be merged? This is making a lot of false-positive errors in my CI/CD process.

[timothycrosley]

This is now merged into development branch and ready for release with 5.0.0. @beneisner, no - that's the exact kind of magic I'm aiming to remove. The good news, is with this latest version you shouldn't need it.