Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pre-commit config and GitHub Actions job #1688

Merged
merged 10 commits into from
Oct 30, 2023
Merged

Conversation

nicholasjng
Copy link
Contributor

Contains the following hooks:

  • buildifier - for formatting and linting Bazel files.
  • mypy, ruff, isort, black - for Python typechecking, import hygiene, static analysis, and formatting.

The pylint CI job was changed to be a pre-commit CI job, where pre-commit is bootstrapped via Python.

Pylint is currently no longer part of the code checks, but can be re-added if requested. The reason to drop was that it does not play nicely with pre-commit, and lots of its functionality and responsibilities are actually covered in ruff.

@dmah42
Copy link
Member

dmah42 commented Oct 27, 2023

looks like pre-commit needs installing.

@dmah42 dmah42 closed this Oct 27, 2023
@dmah42 dmah42 reopened this Oct 27, 2023
@nicholasjng
Copy link
Contributor Author

nicholasjng commented Oct 27, 2023

This is ready for re-review. Not sure why the buildifier fails in CI, it passes on my machine (TM).

Re: integrating clang-format with pre-commit, it seems quite easy, in case you want to restructure / unify the actions: https://stackoverflow.com/questions/55965712/how-do-i-add-clang-formatting-to-pre-commit-hook

EDIT: Need to rebase, one second.

Contains the following hooks:
* buildifier - for formatting and linting Bazel files.
* mypy, ruff, isort, black - for Python typechecking, import hygiene,
static analysis, and formatting.

The pylint CI job was changed to be a pre-commit CI job, where pre-commit
is bootstrapped via Python.

Pylint is currently no longer part of the
code checks, but can be re-added if requested. The reason to drop was
that it does not play nicely with pre-commit, and lots of its
functionality and responsibilities are actually covered in ruff.
@dmah42
Copy link
Member

dmah42 commented Oct 27, 2023

"pylint" is not installed i think

@nicholasjng
Copy link
Contributor Author

Yeah, I forgot deleting that - most (but not all) of pylint's work is taken over by ruff, is it ok with you to drop pylint then? (See the PR comment for my reasons)

I also forgot to add the tool configs to the pyproject.toml - after doing so, I got 31 flags in the report.py, so I am strongly considering excluding that file from typechecking (some crazy stuff is happening in there, which I don't feel like fixing atm).

Barring some type annotations, the stuff I'm about to push should complete this. Are you fine with line length 100 for Python files, or do you require a specific different value?

@dmah42
Copy link
Member

dmah42 commented Oct 27, 2023

Yeah, I forgot deleting that - most (but not all) of pylint's work is taken over by ruff, is it ok with you to drop pylint then? (See the PR comment for my reasons)

if it's a swap out for ruff then sure.

I also forgot to add the tool configs to the pyproject.toml - after doing so, I got 31 flags in the report.py, so I am strongly considering excluding that file from typechecking (some crazy stuff is happening in there, which I don't feel like fixing atm).

sgtm

Barring some type annotations, the stuff I'm about to push should complete this. Are you fine with line length 100 for Python files, or do you require a specific different value?

80 please: https://google.github.io/styleguide/pyguide.html

In particular, set line length 80 for all Python files.
Also ignores the `tools/compare.py` and `tools/gbench/report.py` files
for mypy, since they emit a barrage of errors which we can deal with
later. The errors are mostly related to dynamic classmethod definition.
@nicholasjng
Copy link
Contributor Author

Link to the ruff / pylint comparison: https://docs.astral.sh/ruff/faq/#how-does-ruffs-linter-compare-to-pylint

TLDR; it's not a 1:1 but it covers more rules, and especially when including mypy you more than make up for what you lose on static typechecking.

Python files are reformatted to length 80 now.

@dmah42
Copy link
Member

dmah42 commented Oct 27, 2023

Link to the ruff / pylint comparison: https://docs.astral.sh/ruff/faq/#how-does-ruffs-linter-compare-to-pylint

TLDR; it's not a 1:1 but it covers more rules, and especially when including mypy you more than make up for what you lose on static typechecking.

Python files are reformatted to length 80 now.

sorry, python style is also 2 indents. that should also minimise a lot of the changes.

@nicholasjng
Copy link
Contributor Author

nicholasjng commented Oct 27, 2023

You mean 2 spaces as indent? Alright, that means I will have to remove black again (they only support 4-space indents).

I think I will reset and submit the configurations again, and then run the remaining hooks (i.e. excluding black) over the codebase. That should eliminate the unwanted changes.

Looking at the style guide, it seems indentations are 4 spaces? (https://google.github.io/styleguide/pyguide.html#s3.4-indentation)

I looked at the diff of e.g. report.py, and it seems that a lot of the code is already 4-space indented - am I misunderstanding something here?

It seems to me like black formatting is conforming to the style guide here, the diff is mostly line breaks on reformats on inline lists and tuples, and JSON objects, and of course the switch from single to double quotes.

@dmah42
Copy link
Member

dmah42 commented Oct 30, 2023

You mean 2 spaces as indent? Alright, that means I will have to remove black again (they only support 4-space indents).

I think I will reset and submit the configurations again, and then run the remaining hooks (i.e. excluding black) over the codebase. That should eliminate the unwanted changes.

Looking at the style guide, it seems indentations are 4 spaces? (https://google.github.io/styleguide/pyguide.html#s3.4-indentation)

wow. that's different to the internal version. that'll be fun to integrate later 😅

I looked at the diff of e.g. report.py, and it seems that a lot of the code is already 4-space indented - am I misunderstanding something here?

no i was.

It seems to me like black formatting is conforming to the style guide here, the diff is mostly line breaks on reformats on inline lists and tuples, and JSON objects, and of course the switch from single to double quotes.

@nicholasjng
Copy link
Contributor Author

So regarding clang-format in pre-commit yes or no, what's your take?

Style guide decision is up to you, I don't want to needlessly mess up the repository formatting. Just give me a heads-up on how this should be handled and I'll resubmit accordingly if necessary.

@dmah42
Copy link
Member

dmah42 commented Oct 30, 2023

So regarding clang-format in pre-commit yes or no, what's your take?

Style guide decision is up to you, I don't want to needlessly mess up the repository formatting. Just give me a heads-up on how this should be handled and I'll resubmit accordingly if necessary.

clang-format: not yet.

style guide: follow what the external links say please

@nicholasjng
Copy link
Contributor Author

style guide: follow what the external links say please

https://google.github.io/styleguide/pyguide.html#s1-background says that "Many teams use the black auto-formatter" (which is configured in this PR), so does that make it eligible to use?

From a quick survey of Google Python OSS projects, https://github.com/google/yapf seems to be another popular one - that's also available for setup in pre-commit.

@dmah42
Copy link
Member

dmah42 commented Oct 30, 2023

style guide: follow what the external links say please

https://google.github.io/styleguide/pyguide.html#s1-background says that "Many teams use the black auto-formatter" (which is configured in this PR), so does that make it eligible to use?

From a quick survey of Google Python OSS projects, https://github.com/google/yapf seems to be another popular one - that's also available for setup in pre-commit.

i have no opinion about the tools for Python, as long as the styling is consistent and matches the google style i'm happy.

@nicholasjng
Copy link
Contributor Author

This should be ready to review then. From my best-effort research, the Black style is consistent with the public Google Python Style guide.

@dmah42
Copy link
Member

dmah42 commented Oct 30, 2023

thank you!

@dmah42 dmah42 merged commit b93f5a5 into google:main Oct 30, 2023
53 of 60 checks passed
@nicholasjng nicholasjng deleted the pre-commit branch April 15, 2024 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants