Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use presence of convention-specific sections during docstring inference #3325

Merged
merged 1 commit into from
Mar 3, 2023

Conversation

charliermarsh
Copy link
Member

Previously, if you had a docstring like:

def get_gas_price(data: EasyEnergyData, hours: int) -> float | None:
    """Get the gas price for a given hour.

    Args:
        data: The data object.
        hours: The number of hours to add to the current time.

    Returns:
        The gas market price value.
    """

We'd first check for any NumPy-style sections. Returns is a valid section header for both NumPy- and Google-style docstrings, so we'd infer that this is a NumPy-style docstring, and validate it as such. However, Args is a Google-style header, and isn't part of the NumPy convention.

This PR thus improves the inference logic by treating docstrings that contain Args or Arguments as Google-style, and docstrings that contain Parameters or Other Parameters as NumPy-style. If none of those sections exist, we then user the convention with the most matches (there could be weird and subtle cases here in which users are unintentionally mixing header conventions by accident, but this seems like a strict improvement).

Closes #3321.

@charliermarsh charliermarsh merged commit dedf8aa into main Mar 3, 2023
@charliermarsh charliermarsh deleted the charlie/docstrings branch March 3, 2023 22:13
renovate bot referenced this pull request in ixm-one/pytest-cmake-presets Mar 4, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `^0.0.253` ->
`^0.0.254` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/compatibility-slim/0.0.253)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/confidence-slim/0.0.253)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.254`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.254)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.253...v0.0.254)

<!-- Release notes generated using configuration in .github/release.yml
at main -->

#### What's Changed

##### Rules

- \[`pyupgrade`] Replace tuples with type union in isinstance or
issubclass calls by
[@&#8203;martinlehoux](https://togithub.com/martinlehoux) in
[https://github.com/charliermarsh/ruff/pull/3280](https://togithub.com/charliermarsh/ruff/pull/3280)
- \[`flake8-pyi`] Add flake-pyi PYI033 "Do not use type comments in
stubs" by [@&#8203;konstin](https://togithub.com/konstin) in
[https://github.com/charliermarsh/ruff/pull/3302](https://togithub.com/charliermarsh/ruff/pull/3302)
- \[`flake8-pyi`] PYI006 bad version info comparison by
[@&#8203;konstin](https://togithub.com/konstin) in
[https://github.com/charliermarsh/ruff/pull/3291](https://togithub.com/charliermarsh/ruff/pull/3291)
- \[`pycodestyle`] feat(E251,E252): add rules by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3274](https://togithub.com/charliermarsh/ruff/pull/3274)
- \[`pycodestyle`] feat(E211): add rule + autofix by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3313](https://togithub.com/charliermarsh/ruff/pull/3313)
- \[`pycodestyle`] feat(e225,226,227,228): add rules by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3300](https://togithub.com/charliermarsh/ruff/pull/3300)
- \[`pycodestyle`] feat(W191): add indentation_contains_tabs by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3249](https://togithub.com/charliermarsh/ruff/pull/3249)
- Redirect `RUF004` to `B026` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3293](https://togithub.com/charliermarsh/ruff/pull/3293)

##### CLI

- Add a `--ignore-noqa` CLI flag to force-ignore noqa directives by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3296](https://togithub.com/charliermarsh/ruff/pull/3296)

##### Settings

- Implement `property-decorators` configuration option for pydocstyle by
[@&#8203;staticssleever668](https://togithub.com/staticssleever668) in
[https://github.com/charliermarsh/ruff/pull/3311](https://togithub.com/charliermarsh/ruff/pull/3311)
- Always include `@classmethod` and `@staticmethod` in decorator lists
by [@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3314](https://togithub.com/charliermarsh/ruff/pull/3314)
- Detect quote style ignoring docstrings by
[@&#8203;bz2](https://togithub.com/bz2) in
[https://github.com/charliermarsh/ruff/pull/3306](https://togithub.com/charliermarsh/ruff/pull/3306)

##### Bug Fixes

- Deduplicate SIM116 errors by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3260](https://togithub.com/charliermarsh/ruff/pull/3260)
- Don't flag keyword-based logging format strings by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3261](https://togithub.com/charliermarsh/ruff/pull/3261)
- Avoid raising TRY200 violations within new scopes by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3275](https://togithub.com/charliermarsh/ruff/pull/3275)
- Use expression span for yoda-conditions fixes by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3276](https://togithub.com/charliermarsh/ruff/pull/3276)
- Avoid PEP 585 rewrites when builtins are shadowed by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3286](https://togithub.com/charliermarsh/ruff/pull/3286)
- Ignore unused imports in ModuleNotFoundError blocks by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3288](https://togithub.com/charliermarsh/ruff/pull/3288)
- Treat function type annotations within classes as runtime-required by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3312](https://togithub.com/charliermarsh/ruff/pull/3312)
- Extend `RET503` autofixes to "end of statement", including comments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3324](https://togithub.com/charliermarsh/ruff/pull/3324)
- Handle empty NamedTuple and TypedDict conversions by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3251](https://togithub.com/charliermarsh/ruff/pull/3251)
- Use `identifier_range` for a few more rules by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3254](https://togithub.com/charliermarsh/ruff/pull/3254)
- Match non-lowercase with S105 again by
[@&#8203;scop](https://togithub.com/scop) in
[https://github.com/charliermarsh/ruff/pull/3258](https://togithub.com/charliermarsh/ruff/pull/3258)
- Abort when unable to fix relative imports past module root by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3319](https://togithub.com/charliermarsh/ruff/pull/3319)
- Use presence of convention-specific sections during docstring
inference by [@&#8203;charliermarsh](https://togithub.com/charliermarsh)
in
[https://github.com/charliermarsh/ruff/pull/3325](https://togithub.com/charliermarsh/ruff/pull/3325)
- Treat callables within type definitions as default-non-types by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3329](https://togithub.com/charliermarsh/ruff/pull/3329)
- Explicitly put `Path(...)` in Pathlib violations by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/3333](https://togithub.com/charliermarsh/ruff/pull/3333)

#### New Contributors

- [@&#8203;rouge8](https://togithub.com/rouge8) made their first
contribution in
[https://github.com/charliermarsh/ruff/pull/3277](https://togithub.com/charliermarsh/ruff/pull/3277)
- [@&#8203;staticssleever668](https://togithub.com/staticssleever668)
made their first contribution in
[https://github.com/charliermarsh/ruff/pull/3311](https://togithub.com/charliermarsh/ruff/pull/3311)
- [@&#8203;bz2](https://togithub.com/bz2) made their first contribution
in
[https://github.com/charliermarsh/ruff/pull/3306](https://togithub.com/charliermarsh/ruff/pull/3306)
- [@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) made
their first contribution in
[https://github.com/charliermarsh/ruff/pull/3333](https://togithub.com/charliermarsh/ruff/pull/3333)

**Full Changelog**:
astral-sh/ruff@v0.0.253...v0.0.254

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/ixm-one/pytest-cmake-presets).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xNTQuNyIsInVwZGF0ZWRJblZlciI6IjM0LjE1NC43In0=-->

Signed-off-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@epenet
Copy link
Contributor

epenet commented Mar 7, 2023

Thanks @charliermarsh 👍 🥳

smackesey pushed a commit to dagster-io/dagster that referenced this pull request Mar 10, 2023
### Summary & Motivation

Previously, if Ruff saw a docstring like:

```py
def get_gas_price(data: EasyEnergyData, hours: int) -> float | None:
    """Get the gas price for a given hour.

    Args:
        data: The data object.
        hours: The number of hours to add to the current time.

    Returns:
        The gas market price value.
    """
```

It treated it as a NumPy-style docstring, since `Returns` is a valid
section header in both NumPy- and Google-style conventions (despite
`Args` being a Google-only header).

This matches `pydocstyle`'s own behavior, but I think we can improve it
a bit. Specifically, in astral-sh/ruff#3325,
we now use those convention-specific headers to improve the docstring
convention inference.

I ran `main` over Dagster, and it surfaced a couple issues, since it now
(correctly) treats some docstrings that it _used_ to consider
NumPy-style as Google-style.

As an aside, you can forgo this convention inference altogether via:

```toml
[tool.ruff.pydocstyle]
convention = "google"
```

This is recommended, but there's at least one NumPy style docstring in
`dagster/python_modules/dagster/dagster_tests/general_tests/py3_tests/test_inference.py`,
so I omitted it for now in case that's intentional.

### How I Tested These Changes

`ruff .`
renovate bot referenced this pull request in allenporter/flux-local Mar 13, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `==0.0.253` ->
`==0.0.254` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/compatibility-slim/0.0.253)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.254/confidence-slim/0.0.253)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.254`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.254)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.253...v0.0.254)

<!-- Release notes generated using configuration in .github/release.yml
at main -->

##### What's Changed

##### Rules

- \[`pyupgrade`] Replace tuples with type union in isinstance or
issubclass calls by
[@&#8203;martinlehoux](https://togithub.com/martinlehoux) in
[https://github.com/charliermarsh/ruff/pull/3280](https://togithub.com/charliermarsh/ruff/pull/3280)
- \[`flake8-pyi`] Add flake-pyi PYI033 "Do not use type comments in
stubs" by [@&#8203;konstin](https://togithub.com/konstin) in
[https://github.com/charliermarsh/ruff/pull/3302](https://togithub.com/charliermarsh/ruff/pull/3302)
- \[`flake8-pyi`] PYI006 bad version info comparison by
[@&#8203;konstin](https://togithub.com/konstin) in
[https://github.com/charliermarsh/ruff/pull/3291](https://togithub.com/charliermarsh/ruff/pull/3291)
- \[`pycodestyle`] feat(E251,E252): add rules by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3274](https://togithub.com/charliermarsh/ruff/pull/3274)
- \[`pycodestyle`] feat(E211): add rule + autofix by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3313](https://togithub.com/charliermarsh/ruff/pull/3313)
- \[`pycodestyle`] feat(e225,226,227,228): add rules by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3300](https://togithub.com/charliermarsh/ruff/pull/3300)
- \[`pycodestyle`] feat(W191): add indentation_contains_tabs by
[@&#8203;carlosmiei](https://togithub.com/carlosmiei) in
[https://github.com/charliermarsh/ruff/pull/3249](https://togithub.com/charliermarsh/ruff/pull/3249)
- Redirect `RUF004` to `B026` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3293](https://togithub.com/charliermarsh/ruff/pull/3293)

##### CLI

- Add a `--ignore-noqa` CLI flag to force-ignore noqa directives by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3296](https://togithub.com/charliermarsh/ruff/pull/3296)

##### Settings

- Implement `property-decorators` configuration option for pydocstyle by
[@&#8203;staticssleever668](https://togithub.com/staticssleever668) in
[https://github.com/charliermarsh/ruff/pull/3311](https://togithub.com/charliermarsh/ruff/pull/3311)
- Always include `@classmethod` and `@staticmethod` in decorator lists
by [@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3314](https://togithub.com/charliermarsh/ruff/pull/3314)
- Detect quote style ignoring docstrings by
[@&#8203;bz2](https://togithub.com/bz2) in
[https://github.com/charliermarsh/ruff/pull/3306](https://togithub.com/charliermarsh/ruff/pull/3306)

##### Bug Fixes

- Deduplicate SIM116 errors by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3260](https://togithub.com/charliermarsh/ruff/pull/3260)
- Don't flag keyword-based logging format strings by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3261](https://togithub.com/charliermarsh/ruff/pull/3261)
- Avoid raising TRY200 violations within new scopes by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3275](https://togithub.com/charliermarsh/ruff/pull/3275)
- Use expression span for yoda-conditions fixes by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3276](https://togithub.com/charliermarsh/ruff/pull/3276)
- Avoid PEP 585 rewrites when builtins are shadowed by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3286](https://togithub.com/charliermarsh/ruff/pull/3286)
- Ignore unused imports in ModuleNotFoundError blocks by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3288](https://togithub.com/charliermarsh/ruff/pull/3288)
- Treat function type annotations within classes as runtime-required by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3312](https://togithub.com/charliermarsh/ruff/pull/3312)
- Extend `RET503` autofixes to "end of statement", including comments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3324](https://togithub.com/charliermarsh/ruff/pull/3324)
- Handle empty NamedTuple and TypedDict conversions by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3251](https://togithub.com/charliermarsh/ruff/pull/3251)
- Use `identifier_range` for a few more rules by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3254](https://togithub.com/charliermarsh/ruff/pull/3254)
- Match non-lowercase with S105 again by
[@&#8203;scop](https://togithub.com/scop) in
[https://github.com/charliermarsh/ruff/pull/3258](https://togithub.com/charliermarsh/ruff/pull/3258)
- Abort when unable to fix relative imports past module root by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3319](https://togithub.com/charliermarsh/ruff/pull/3319)
- Use presence of convention-specific sections during docstring
inference by [@&#8203;charliermarsh](https://togithub.com/charliermarsh)
in
[https://github.com/charliermarsh/ruff/pull/3325](https://togithub.com/charliermarsh/ruff/pull/3325)
- Treat callables within type definitions as default-non-types by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[https://github.com/charliermarsh/ruff/pull/3329](https://togithub.com/charliermarsh/ruff/pull/3329)
- Explicitly put `Path(...)` in Pathlib violations by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[https://github.com/charliermarsh/ruff/pull/3333](https://togithub.com/charliermarsh/ruff/pull/3333)

##### New Contributors

- [@&#8203;rouge8](https://togithub.com/rouge8) made their first
contribution in
[https://github.com/charliermarsh/ruff/pull/3277](https://togithub.com/charliermarsh/ruff/pull/3277)
- [@&#8203;staticssleever668](https://togithub.com/staticssleever668)
made their first contribution in
[https://github.com/charliermarsh/ruff/pull/3311](https://togithub.com/charliermarsh/ruff/pull/3311)
- [@&#8203;bz2](https://togithub.com/bz2) made their first contribution
in
[https://github.com/charliermarsh/ruff/pull/3306](https://togithub.com/charliermarsh/ruff/pull/3306)
- [@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) made
their first contribution in
[https://github.com/charliermarsh/ruff/pull/3333](https://togithub.com/charliermarsh/ruff/pull/3333)

**Full Changelog**:
astral-sh/ruff@v0.0.253...v0.0.254

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xNjAuMCIsInVwZGF0ZWRJblZlciI6IjM0LjE2MC4wIn0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

D406 auto-detect pydocstyle
2 participants