STYLE Increase coverage of inconsistent-namespace-usage

[MarcoGorelli]

If you already have experience contributing, then please leave this issue to newcomers - thanks 👍

---

In .pre-commit-config.yaml, the check inconsistent-namespace-usage is limited to pandas/tests/frame/. Let's expand it to the rest of pandas/tests/.

The task here is:

0. make sure you're familiar with the contributing guide
1. choose a folder from the list below (e.g. series) and add it to the check in .pre-commit-config.yaml. E.g., if you chose series, that would be:

    -   id: inconsistent-namespace-usage
        name: 'Check for inconsistent use of pandas namespace in tests'
        entry: python scripts/check_for_inconsistent_pandas_namespace.py
        language: python
        types: [python]
        files: ^pandas/tests/(frame|series)/

2. temporarily add tokenize-rt as an additional dependency and --replace as an arg (these will be removed later):

    -   id: inconsistent-namespace-usage
        name: 'Check for inconsistent use of pandas namespace in tests'
        entry: python scripts/check_for_inconsistent_pandas_namespace.py
        language: python
        types: [python]
        files: ^pandas/tests/(frame|series)/
        additional_dependencies: [tokenize-rt]
        args: [--replace]

3. run this check on all files with pre-commit, i.e.:

  $ pre-commit run inconsistent-namespace-usage --all-files
  [INFO] Initializing environment for local:tokenize-rt.
  [INFO] Installing environment for local.
  [INFO] Once installed this environment will be reused.
  [INFO] This may take a few minutes...
  Check for inconsistent use of pandas namespace in tests.......................Failed
  - hook id: inconsistent-namespace-usage
  - files were modified by this hook

4. run it again - this time, it should pass

$ pre-commit run inconsistent-namespace-usage --all-files
Check for inconsistent use of pandas namespace in tests.......................Passed

Check that the other hooks still pass - if you enable pre-commit (pre-commit install), then this will happen automatically when you try to commit your changes (see step 6)

5. revert the changes to .pre-commit-config.yaml (we'll update it once these changes are all so as to avoid a barrage of merge conflicts)
6. commit your changes, and open a pull request. Check over your changes, make sure they look sensible, let me know if not

---

Here are the folders in pandas/tests which this check needs expanding to:

• series
• scalar
• resample
• util
• tseries
• dtypes
• arrays
• plotting
• tools
• indexing
• strings
• reductions
• frame
• arithmetic
• config
• api
• generic
• io
• base
• window
• internals
• reshape
• computation
• apply
• groupby
• tslibs
• libs
• indexes
• extension

No need to ask for permission to work on this, just leave a comment letting people know which folder(s) from the list above you're working on

[alexprincel]

I ran the script for "extension" and it made no change to files in that folder. I checked my procedure against your example above to make sure this was not the result of a bad manipulation. When changing to series I get the same output as in your post.

[alexprincel]

Currently working on indexes, will submit a PR as there were changes on that folder.

[jorisvandenbossche]

@MarcoGorelli the automatic script only changes it from pd.DataFrame(..) to DataFrame(..) and not the other way around? (as previously, we only agreed to be consistent within a file, not necessarily to change all files to use the non-pd. way)

[MarcoGorelli]

@jorisvandenbossche if there's both, it'll choose the non-pd. one. If the file is already consistently using the pd. one, then it'll leave it untouched.

e.g.

import pandas as pd

def test_0():
    pd.DataFrame()

def test_1():
    pd.DataFrame()

would be left untouched (i.e. it would keep being consistent within the file), but

import pandas as pd
from pandas import DataFrame

def test_0():
    pd.DataFrame()

def test_1():
    DataFrame()

would become

import pandas as pd
from pandas import DataFrame

def test_0():
    DataFrame()

def test_1():
    DataFrame()

[alexprincel]

libs and tslibs passed without fixes,

[alexprincel]

computation passed without fixes

[01-vyom]

window, config, generic, and internals passed without fixes

Working on api and arithmetic

[sionedbaker]

As part of pyladies-sprint-March-2021, I looked at dir reductions

[MarcoGorelli]

Nice, thanks @sionedbaker ! It doesn't look like you've opened a pull request though, let us know if you want//need help with that

[Varun270]

I am new to open source contribution and I have cloned this repo in my system but I am unable to understand what are the corrections needed to be done. Can anyone help me, I am literally so confused.

[deepang17]

working on tools

[MarcoGorelli]

There's only 9 files left now, so if anyone wants to close this issue then please change files to ^pandas/tests/ (though revert the changes to args and additional_dependencies before committing) - then we can close this issue

[deepang17]

Working to close this issue

[deepang17]

As you can see in the above image that by replacing pd.unique(mindex) to unique(mindex) we get F821 undefined name 'unique' error.

pandas/pandas/tests/test_algos.py

Line 494 in 70c91a2

result = pd.unique(mindex)

Rest all files are resolved.

[deepang17]

One solution is to exclude test_algos.py from checking for inconsistency in pre-commit.
exclude: ^pandas/tests/test_algos.py$

[MarcoGorelli]

There's only 9 files left now, so if anyone wants to close this issue then please change files to ^pandas/tests/ (though revert the changes to args and additional_dependencies before committing) - then we can close this issue

My bad, I missed that @sionedbaker hasn't submitted her PR for reductions yet. @sionedbaker , are you still working on this? If so, let's wait before merging #40286

[sionedbaker]

Hi @MarcoGorelli , very sorry for taking so long to reply, but I had huge problems with pushing my changes to github (due to me having to setup 2FA but I've push changes to my fork now, and then created PR: #40310
I'm not sure if @deepang17 has already solved this, therefore no worries if my PR is surplus to requirements. I very much enjoyed the session on Saturday, but I probably need more practise with submitting this via github as I've found that part the trickiest

[MarcoGorelli]

No worries, thanks for your PR and happy IWD! I recommend the first 3 chapters of the pro-git book, the rest is just practice

[sionedbaker]

Many thanks @MarcoGorelli and thanks for approving the PR. Thanks for the git book, looks good, be really interested in working through some (very simple!) panda issues in the next few weeks, I will keep an eye on the github issues. Be really interested in a further follow up session so will try to keep up to date with pyladies. thanks for all your help