Fix python linter inconsistent behaviour with quotes

[li-boxuan]

This has been a headache for a long time, and we had #1071 and #1100 with the hope to fix the inconsistent behaviour across linters and environments. However, we recently found out that double-quote-string-fixer plugin in pre-commit-hook has inconsistent behaviour on python 3.11 and 3.12. See discussion here. This is sad because while this plugin enforces single quote behaviour with 3.11, it doesn't always enforce so with 3.12. Specifically, with f"str" syntax, this plugin allows both single quotes and double quotes with python 3.12.

The problem is, some developers have black linter installed/integrated with their IDE, which is probably the most popular linter in python world (ranked by GitHub stars). This linter insists on always using double quotes. Now we have black and double-quote-string-fixer fight each other (iff the developer uses python 3.12) for some quotes (f"str" syntax).

After a lot of research, I couldn't find a way to enforce single quote behaviour without introducing a new dependency, flake8, together with a plugin for it to enforce quotes' behavior. The benefit of flake8 is it allows you to either enforce single quotes or double quotes, while black only allows you to enforce double quotes.

To avoid disruptions to ongoing PRs and tons of merge conflicts, this PR continues to enforce single quote behavior by using flake8 rather than double-quote-string-fixer. In addition, this PR adds a black config to pyproject.toml so that anyone using black formatter in their IDE now wouldn't attempt to change single quotes back to double quotes. Even if they do so, CI would reject it, but it's better for it not to happen at the first place.

[rbren]

Ugh this is painful 🙃

If we do all the files at once, like in this PR, it will cause a lot of merge conflict pain. In the past, we've only enforced the linter on changed files, which let us do a gradual roll out--is that possible here?

[li-boxuan]

If we do all the files at once, like in this PR, it will cause a lot of merge conflict pain.

Yeah that's my concern too.

In the past, we've only enforced the linter on changed files, which let us do a gradual roll out

Ah yeah that makes sense. I could let CI to enforce the same behavior too.

That will make the same amount of pain, and let the pain last longer. It will also make a bunch of PRs hard to review. The burden is shifted from developers to reviewers. Plus, developers concurrently working on same file(s) would still need to resolve conflicts.

[li-boxuan]

Maybe I shall continue to enforce the current behaviour to avoid annoying merge conflicts - using single quotes over double quotes. I think flake8 can do that.

[iFurySt]

i'm on the black side 😭 actually, we discussed the formatter the first time was decided to use black, but seems don't force use :(

[li-boxuan]

i'm on the black side 😭 actually, we discussed the formatter the first time was decided to use black, but seems don't force use :(

I understand your frustration 😭 but it is what it is. In hindsight, we should force that in CI as soon as the community decides a format to use. The later, the larger resistance and pain.

What I am gonna do now:

1. Use flake8 to enforce single quotes (pre-commit-hook is not reliable enough to enforce a consistent behaviour)
2. Add a VSCode config to enable black's skip-string-normalization option, i.e. stop converting single quotes to double quotes.

If the above attempt works, then we can enforce a consistent behaviour and do minimal abruption to existing works.

If we want to start using black, IMHO a maintainer shall help existing open PRs to resolve the conflicts to make everyone happy (I don't have write access).

[iFurySt]

i'm on the black side 😭 actually, we discussed the formatter the first time was decided to use black, but seems don't force use :(

I understand your frustration 😭 but it is what it is. In hindsight, we should force that in CI as soon as the community decides a format to use. The later, the larger resistance and pain.

What I am gonna do now:

1. Use flake8 to enforce single quotes (pre-commit-hook is not reliable enough to enforce a consistent behaviour)
2. Add a VSCode config to enable black's skip-string-normalization option, i.e. stop converting single quotes to double quotes.

If the above attempt works, then we can enforce a consistent behaviour and do minimal abruption to existing works.

If we want to start using black, IMHO a maintainer shall help existing open PRs to resolve the conflicts to make everyone happy (I don't have write access).

IMO, we need to add make test or make lint to Makefile, this can do all test/lint jobs locally, helping contributors to save time. i can cooperate with you if you need.

[li-boxuan]

IMO, we need to add make test or make lint to Makefile, this can do all test/lint jobs locally, helping contributors to save time

Good idea. I imagine that would be helpful to those who (for some reason) disable the pre-commit-hook. Should be an one-liner change. Let me quickly do that. Thx!