[ci] [python-package] enable ruff-format on all Python code

[jameslamb]

Contributes to #6304.

Proposes enforcing ruff-format on all of the Python code checked into source control in this project.

This only has changes in python-package/ because previous PRs touched other files:

• [ci] enable ruff-format on some files, add pre-commit config #6308
• [ci] [python-package] enable ruff-format on tests and examples #6317

Notes for reviewers

I made a few manual changes. I've added comments on the relevant parts of the diff explaining those.

After this, I'll put up one more PR with a .git-blame-ignore-revs file, dropping all these large reformatting changes from the git blame.

Relevant docs:

• https://github.com/astral-sh/ruff-pre-commit
• https://docs.astral.sh/ruff/formatter/black/

[jameslamb]

Commenting on this part of the diff to share a general point.... I added many of the trailing commas in this PR manually.

Without that trailing command, ruff / black wants to turn this:

def __init_from_csc(
    self,
    csc: scipy.sparse.csc_matrix,
    params_str: str,
    ref_dataset: Optional[_DatasetHandle]
) -> "Dataset"

Into this:

def __init_from_csc(
    self, csc: scipy.sparse.csc_matrix, params_str: str, ref_dataset: Optional[_DatasetHandle]
) -> "Dataset"

For details on that, see https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#the-magic-trailing-comma.

And this:

LightGBM/python-package/pyproject.toml

Line 121 in 1b792e7

skip-magic-trailing-comma = false

[jameslamb]

I manually pulled this out into a new type, LGBM_SetFieldType, because ruff wanted to split this into a one-type-per-line list, which I thought made the function signature difficult to read.

[jameslamb]

I did this one manually, for consistency with all the other has-more-than-1-argument functions.

[jameslamb]

I accidentally made this change manually, while trying to revert ruff's change to the other __call__() call a bit further down... but I think we should keep this change. That accident just revealed that these were inconsistent with each other, and I like the one {argument}: {type} per line style.

[jameslamb]

Thanks @guolinke ! I'd really like @jmoralez 's opinion on this one as well before we merge.

[jmoralez]

Thanks!

[jameslamb]

Thanks so much @jmoralez @borchero @guolinke !