Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandas.DataFrame.from_dict : issue with dtype and columns arguments types #896

Closed
mutricyl opened this issue Mar 29, 2024 · 2 comments · Fixed by #897
Closed

pandas.DataFrame.from_dict : issue with dtype and columns arguments types #896

mutricyl opened this issue Mar 29, 2024 · 2 comments · Fixed by #897
Labels
Constructors Series/DataFrame/Index/pd.array Constructors DataFrame DataFrame data structure

Comments

@mutricyl
Copy link
Contributor

Describe the bug
issue with types for pandas.DataFrame.from_dict method:

  • dtype is restricted to str in the stub file when pands allows for str, type and None
  • columns is restricted to list of strings when pandas allows for much more (at least: str, int, float, datetime)

To Reproduce

  1. minimal exemple:
import pandas as pd
import datetime

data = {'l1': [1, 2, 3],
        'l2': [4, 5, 6]}

# all of the following are accepted by pandas
# however mypy complains with the last 3
pd.DataFrame.from_dict(data, orient='index')  # OK
pd.DataFrame.from_dict(data, orient='index', dtype='float')  # OK
pd.DataFrame.from_dict(data, orient='index', columns=['a', 'b', 'c'])  # OK
pd.DataFrame.from_dict(data, orient='index', dtype=float)  # NOK
pd.DataFrame.from_dict(data, orient='index', dtype=None)  # NOK
pd.DataFrame.from_dict(data, orient='index',
                       columns=[1.0, 2, datetime.datetime.now()])  # NOK
  1. type checker : mypy
  2. typing errors:
from_dict_test.py:12: error: Argument "dtype" to "from_dict" of "DataFrame" has incompatible type "type[float]"; expected "str"  [arg-type]
from_dict_test.py:13: error: Argument "dtype" to "from_dict" of "DataFrame" has incompatible type "None"; expected "str"  [arg-type]
from_dict_test.py:15: error: List item 0 has incompatible type "float"; expected "str"  [list-item]
from_dict_test.py:15: error: List item 1 has incompatible type "int"; expected "str"  [list-item]
from_dict_test.py:15: error: List item 2 has incompatible type "datetime"; expected "str"  [list-item]

Please complete the following information:

  • OS: Windows
  • Windows 10 22H2 build: 19045.4170
  • Python 3.12.2 | packaged by Anaconda, Inc. | (main, Feb 27 2024, 17:28:07) [MSC v.1916 64 bit (AMD64)] on win32
  • mypy 1.9.0
  • pandas-stubs 2.2.0.240218
@Dr-Irv
Copy link
Collaborator

Dr-Irv commented Mar 29, 2024

Thanks for the report.

There are multiple issues with the stub for DataFrame.from_dict().

  • Argument dtype should be AsTypeArg | None
  • Argument columns should be Axes | None, but only if orient==Literal["index"]

So those updates could be done to the stubs, with an appropriate overload to handle the columns argument only being accepted for orient=="index" .

PR with tests welcome.

@Dr-Irv Dr-Irv added Constructors Series/DataFrame/Index/pd.array Constructors DataFrame DataFrame data structure labels Mar 29, 2024
@mutricyl
Copy link
Contributor Author

I think I can handle that, I'll give it a try anyway. Thanks for the tips

mutricyl pushed a commit to mutricyl/pandas-stubs that referenced this issue Mar 29, 2024
Dr-Irv pushed a commit that referenced this issue Apr 1, 2024
* tentative solution for #896

* remove Unnecessary "# pyright: ignore" rule error

* improving dataframe.from_dict test

* check(assert_type every test

* remove unwanted ellipsis

* Adding overload and test to cover when litteral is not used

* Revert "Adding overload and test to cover when litteral is not used"

This reverts commit f760452.

---------

Co-authored-by: Laurent Mutricy <laurent.mutricy@ekium.eu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Constructors Series/DataFrame/Index/pd.array Constructors DataFrame DataFrame data structure
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants