Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support duplicate-code message with --jobs #8757

Merged

Conversation

jacobtylerwalls
Copy link
Member

Type of Changes

Type
βœ“ πŸ› Bug fix

Description

All the groundwork for detecting duplicate-code with --jobs was laid in #4007, but it was only unit tested without actually checking that messages were emitted.

Closes #374 (I moved the remaining cyclic-import issue to #4171)

@jacobtylerwalls jacobtylerwalls added this to the 3.0.0a7 milestone Jun 10, 2023
@jacobtylerwalls jacobtylerwalls changed the title Support duplicate-code message with --jobs Support duplicate-code message with --jobs Jun 10, 2023
@@ -233,7 +233,6 @@ def test_parallel_execution(self) -> None:
join(HERE, "functional", "a", "arguments.py"),
],
out=out,
# We expect similarities to fail and an error
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reason to expect similarity to fail when checking only one file. Maybe an earlier version of this test checked the same file twice?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment was very unclear, sorry about that.

It was meant to say something like:

if we test the same file twice, which this test was doing, in parallel mode we get extra errors, so we only test a single file here in parallel mode, ensuring it errors. All parallel checks are in their own file now.

@codecov
Copy link

codecov bot commented Jun 10, 2023

Codecov Report

Merging #8757 (dc5f010) into main (2acca90) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head dc5f010 differs from pull request most recent head 0a88e97. Consider uploading reports for the commit 0a88e97 to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #8757   +/-   ##
=======================================
  Coverage   95.82%   95.82%           
=======================================
  Files         173      173           
  Lines       18386    18386           
=======================================
+ Hits        17618    17619    +1     
+ Misses        768      767    -1     
Impacted Files Coverage Ξ”
pylint/lint/parallel.py 92.85% <ΓΈ> (ΓΈ)
pylint/testutils/_primer/primer_run_command.py 32.30% <ΓΈ> (+0.48%) ⬆️
pylint/checkers/similar.py 96.28% <100.00%> (+<0.01%) ⬆️

@@ -130,7 +130,7 @@ def check_parallel(
"""Use the given linter to lint the files with given amount of workers (jobs).

This splits the work filestream-by-filestream. If you need to do work across
multiple files, as in the similarity-checker, then implement the map/reduce mixin functionality.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it no longer [considered] a mix-in class?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, see #6383 and #8409

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oooooh, hadn't seen those, sexy

"""
Similar.combine_mapreduce_data(self, linesets_collection=data)
self.close()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple as that is it! Great stuff!! :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holy mother of Guido, was it that simple ?!

@jacobtylerwalls
Copy link
Member Author

jacobtylerwalls commented Jun 10, 2023

I just want to see the primer result with --duplicate-code back in, but it doesn't relate to --jobs, so I'll probably drop b74b96d in a minute and coverage will be happy again.

Copy link
Collaborator

@DanielNoord DanielNoord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't looked at the code but great find Jacob!

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is such an elegant fix to a problem that have been plaguing pylint for a decade, amazing ! Thanks also to @doublethefish who did almost all the work minus that one self.close() in 2021πŸ˜„

@doublethefish
Copy link
Contributor

IIRC I tried adding .close() but it didn't work then. I suspect other changes/improvements have helped here. It's very good to see done however :)

@jacobtylerwalls jacobtylerwalls merged commit 1ef3b0b into pylint-dev:main Jun 10, 2023
40 checks passed
@jacobtylerwalls jacobtylerwalls deleted the duplicate-code-jobs branch June 10, 2023 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Different output with --jobs=1 and --jobs=2
4 participants