Support globbing patterns for source-roots

[alexey-pelykh]

Add globbing pattern support for source-roots.

Type of Changes

	Type
✓	✨ New feature

Description

Closes #8290

[ollie-iterators]

#8286 fixed the issue with self-import

[codecov]

Codecov Report

Merging #8281 (f3d3e55) into main (e64f043) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8281      +/-   ##
==========================================
- Coverage   95.46%   95.46%   -0.01%     
==========================================
  Files         177      177              
  Lines       18704    18703       -1     
==========================================
- Hits        17856    17855       -1     
  Misses        848      848              

Impacted Files	Coverage Δ
pylint/lint/base_options.py	100.00% <ø> (ø)
pylint/pyreverse/main.py	93.87% <ø> (ø)
pylint/config/argument.py	100.00% <100.00%> (ø)
pylint/testutils/reporter_for_tests.py	93.02% <0.00%> (-0.32%)	⬇️

[DanielNoord]

I haven't read much of the discussion but was triggered about the comment about breaking changes in the accompanying issue. Note that the current implementation is definitely a breaking change as it changes the behaviour of an existing option. That's what we normally consider to be a breaking change.

[alexey-pelykh]

@DanielNoord it's not a breaking change since it's backwards compatible and on top the option in question hasn't been released yet as well

[alexey-pelykh]

@Pierre-Sassoulas not a WIP anymore, yet there's a regression in test coverage since _paths_csv_transformer is no longer used. I'm not sure it's worth removing it really yet I'm open for options

Also, it would be much appreciated if this will go into the same release as the source-roots feature 🙏

[DudeNr33]

I really like the idea behind this.
However I share Pierre's opinion that this can get quite confusing for users now as this would only be available for the new --source-roots option, but not for others.

Users that learn glob patterns work for --source-roots might try to use the same for other options like --ignore-paths or simply when calling pylint (you could think of pylint --recursive=y src/prefix**/*.py or something similar) and spend quite some time trying to figure out why glob patterns work for one option but not for another.

A more holistic approach that takes into account the other options/arguments that accept paths would be better in my opinion.

[alexey-pelykh]

@DudeNr33 I agree and adding globbing support for the input sources is easy and non-breaking, yet ignore-paths e.g. are regexp-based right now so that would be breaking unless we add a new set of parameters. What would be your vision on overcoming that?

[DudeNr33]

@DudeNr33 I agree and adding globbing support for the input sources is easy and non-breaking, yet ignore-paths e.g. are regexp-based right now so that would be breaking unless we add a new set of parameters. What would be your vision on overcoming that?

My take on this would be:

• support globbing where the current behaviour is not altered (i.e. anything that is not regex based probably)
• leave the regex based options as they are for v2.x
• change the default behavior from regex to globbing in v3.0, optionally adding support for people who really must use regex patterns by adding --ignore-paths-regex etc.
• nice to have in v2.x: warn the user if an expression that is currently regex based cannot be evaluated as glob pattern, and make them aware that they must use regex pattern here

@DanielNoord your opinion?

[DanielNoord]

I think it is better to do globbing now for those settings where it is not a breaking change (this one). It just makes it easier for users to transition to 3.0 as they would need to update one less config option.

[alexey-pelykh]

@DudeNr33 @DanielNoord @Pierre-Sassoulas so, I've created #8310 and #8311 that can be addressed separately and would allow to unblock this PR.

[alexey-pelykh]

I guess I need some assistance with taming the CI as it fails randomly 🤔

[DudeNr33]

I re-ran the CI jobs and they passed now.
I only got one minor remark on the changelog entry, but that is not a must.

[alexey-pelykh]

Thanks, I've merged the suggestion

[github-actions]

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit f3d3e55

[DudeNr33]

Thanks!

@Pierre-Sassoulas with the follow-up tasks for 3.0 created I think we can put this MR in the 2.17 release. I'm pinging you as you originally set the 3.0 milestone.