TYP: Typing hints in pandas/io/formats/{css,csvs}.py #30398

ShaharNaveh · 2019-12-22T10:53:51Z

pandas/io/formats/css.py

pandas/io/formats/csvs.py

pandas/io/formats/css.py

pandas/io/formats/csvs.py

WillAyd · 2019-12-23T18:39:00Z

pandas/io/formats/csvs.py

-        na_rep="",
+        path_or_buf: Optional[FilePathOrBuffer[str]] = None,
+        sep: str = ",",
+        na_rep: str = "",
        float_format=None,
        cols=None,


Suggested change

cols=None,

cols: Sequence[Hashable] = None,

Any suggestions?

mypy's log for 3 diffrent annotations:

cols: Sequence[Hashable] = None

$ mypy pandas pandas/io/formats/csvs.py:34: error: Incompatible default for argument "cols" (default has type "None", argument has type "Sequence[Hashable]") pandas/core/generic.py:3171: error: Argument "cols" to "CSVFormatter" has incompatible type "Optional[Sequence[Optional[Hashable]]]"; expected "Sequence[Hashable]" Found 2 errors in 2 files (checked 816 source files)

cols: Optional[Sequence[Hashable]] = None

$ mypy pandas pandas/io/formats/csvs.py:128: error: Argument 1 to "list" has incompatible type "Optional[Sequence[Hashable]]"; expected "Iterable[Hashable]" pandas/io/formats/csvs.py:139: error: Argument 1 to "len" has incompatible type "Optional[Sequence[Hashable]]"; expected "Sized" pandas/core/generic.py:3171: error: Argument "cols" to "CSVFormatter" has incompatible type "Optional[Sequence[Optional[Hashable]]]"; expected "Optional[Sequence[Hashable]]" Found 3 errors in 2 files (checked 816 source files)

cols: Optional[Sequence[Optional[Hashable]]] = None

$ mypy pandas pandas/io/formats/csvs.py:128: error: Argument 1 to "list" has incompatible type "Optional[Sequence[Optional[Hashable]]]"; expected "Iterable[Optional[Hashable]]" pandas/io/formats/csvs.py:139: error: Argument 1 to "len" has incompatible type "Optional[Sequence[Optional[Hashable]]]"; expected "Sized" Found 2 errors in 2 files (checked 816 source files)

It looks like the annotations for columns might not be consistent in the IO methods. Can you open up a follow up issue for this and just leave unannotated for now? I think might be more difficult so OK to branch off given there's already a good deal in this PR

pandas/io/formats/csvs.py

WillAyd · 2019-12-23T18:40:57Z

pandas/io/formats/csvs.py

        escapechar=None,
-        decimal=".",
+        decimal: Optional[str] = ".",


Suggested change

decimal: Optional[str] = ".",

decimal: str = ".",

I don't think this is optional

Should it still be typed as

decimal: Optional[str] = "."

?

mypy's log:

$ mypy pandas pandas/core/generic.py:3181: error: Argument "decimal" to "CSVFormatter" has incompatible type "Optional[str]"; expected "str" Found 1 error in 1 file (checked 816 source files)

Same comment

WillAyd · 2019-12-23T18:41:16Z

pandas/io/formats/csvs.py

+        quoting: Optional[int] = None,
+        line_terminator: Optional[str] = "\n",
+        chunksize: Optional[int] = None,
+        quotechar: Optional[str] = '"',


Suggested change

quotechar: Optional[str] = '"',

quotechar: str = '"',

I don't think this is optional

Should it still be typed as

quotechar: Optional[str] = '"'

?

mypy's log:

$ mypy pandas pandas/io/formats/csvs.py:88: error: Incompatible types in assignment (expression has type "None", variable has type "str") Found 1 error in 1 file (checked 816 source files)

I think similar comment as line_terminator

WillAyd · 2019-12-23T18:41:27Z

pandas/io/formats/csvs.py

-        date_format=None,
-        doublequote=True,
+        quoting: Optional[int] = None,
+        line_terminator: Optional[str] = "\n",


Suggested change

line_terminator: Optional[str] = "\n",

line_terminator: str = "\n",

Don't think optional

Should it still be typed as

line_terminator: Optional[str] = "\n"

?

mypy's log:

$ mypy pandas pandas/core/generic.py:3164: error: Argument "line_terminator" to "CSVFormatter" has incompatible type "Optional[str]"; expected "str" Found 1 error in 1 file (checked 816 source files)

Hmm there is a minor disconnect between to_csv and the CSVFormatter class on this. Can you create a follow up issue? Not sure of easiest way to correct

pandas/io/formats/csvs.py

WillAyd

OK for any of the errors we just have a disconnect between defaults for CSVFormatter and to_csv. These may or may not necessarily be issues but just separate out and if you can open up a follow up issue to address would be ideal

WillAyd · 2019-12-24T16:38:58Z

pandas/io/formats/csvs.py

-        na_rep="",
+        path_or_buf: Optional[FilePathOrBuffer[str]] = None,
+        sep: str = ",",
+        na_rep: str = "",
        float_format=None,
        cols=None,


It looks like the annotations for columns might not be consistent in the IO methods. Can you open up a follow up issue for this and just leave unannotated for now? I think might be more difficult so OK to branch off given there's already a good deal in this PR

pandas/io/formats/csvs.py

WillAyd · 2019-12-24T16:43:05Z

pandas/io/formats/csvs.py

-        date_format=None,
-        doublequote=True,
+        quoting: Optional[int] = None,
+        line_terminator: Optional[str] = "\n",


Hmm there is a minor disconnect between to_csv and the CSVFormatter class on this. Can you create a follow up issue? Not sure of easiest way to correct

WillAyd · 2019-12-24T16:43:23Z

pandas/io/formats/csvs.py

+        quoting: Optional[int] = None,
+        line_terminator: Optional[str] = "\n",
+        chunksize: Optional[int] = None,
+        quotechar: Optional[str] = '"',


I think similar comment as line_terminator

WillAyd · 2019-12-24T16:43:36Z

pandas/io/formats/csvs.py

        escapechar=None,
-        decimal=".",
+        decimal: Optional[str] = ".",


Same comment

WillAyd · 2019-12-26T14:09:54Z

@simonjayhawkins care to look?

simonjayhawkins · 2019-12-26T15:18:38Z

pandas/io/formats/csvs.py

+        index_label: Optional[Union[bool, Hashable, Sequence[Hashable]]] = None,
+        mode: str = "w",
+        encoding: Optional[str] = None,
+        compression: Union[str, Mapping[str, str], None] = "infer",


Suggested change

compression: Union[str, Mapping[str, str], None] = "infer",

compression: Optional[Union[str, Mapping[str, str]]] = "infer",

simonjayhawkins · 2019-12-26T15:22:13Z

pandas/io/formats/csvs.py

-        encoding=None,
-        compression="infer",
-        quoting=None,
+        header: Union[bool, Sequence[Hashable]] = True,


question for @WillAyd - do we want Sequence[Optional[Hashable]]] or is this for readability and only add Optional when mypy complains?

Personal preference for only when complaining but it isn't strong if you feel otherwise

fine with that. (although may need to make exception for public apis)

jreback · 2019-12-27T16:02:54Z

thanks @MomIsBestFriend

…ndexing-1row-df * upstream/master: (333 commits) CI: troubleshoot Web_and_Docs failing (pandas-dev#30534) WARN: Ignore NumbaPerformanceWarning in test suite (pandas-dev#30525) DEPR: camelCase in offsets, get_offset (pandas-dev#30340) PERF: implement scalar ops blockwise (pandas-dev#29853) DEPR: Remove Series.compress (pandas-dev#30514) ENH: Add numba engine for rolling apply (pandas-dev#30151) [ENH] Add to_markdown method (pandas-dev#30350) DEPR: Deprecate pandas.np module (pandas-dev#30386) ENH: Add ignore_index for df.drop_duplicates (pandas-dev#30405) BUG: The setting xrot=0 in DataFrame.hist() doesn't work with by and subplots pandas-dev#30288 (pandas-dev#30491) CI: Fix GBQ Tests (pandas-dev#30478) Bug groupby quantile listlike q and int columns (pandas-dev#30485) ENH: Add ignore_index for df.sort_values and series.sort_values (pandas-dev#30402) TYP: Typing hints in pandas/io/formats/{css,csvs}.py (pandas-dev#30398) BUG: raise on non-hashable Index name, closes pandas-dev#29069 (pandas-dev#30335) Replace "foo!r" to "repr(foo)" syntax pandas-dev#29886 (pandas-dev#30502) BUG: preserve EA dtype in transpose (pandas-dev#30091) BLD: add check to prevent tempita name error, clsoes pandas-dev#28836 (pandas-dev#30498) REF/TST: method-specific files for test_append (pandas-dev#30503) marked unused parameters (pandas-dev#30504) ...

ShaharNaveh force-pushed the TYP-io-formats-css/csvs branch from fb88f60 to b7d395a Compare December 22, 2019 11:05

gfyoung added the Typing type annotations, mypy/pyright type checking label Dec 22, 2019

WillAyd requested changes Dec 23, 2019

View reviewed changes

pandas/io/formats/css.py Outdated Show resolved Hide resolved

pandas/io/formats/css.py Outdated Show resolved Hide resolved

pandas/io/formats/csvs.py Outdated Show resolved Hide resolved

pandas/io/formats/csvs.py Outdated Show resolved Hide resolved

ShaharNaveh force-pushed the TYP-io-formats-css/csvs branch from b7d395a to c2d3472 Compare December 23, 2019 09:18

ShaharNaveh requested a review from WillAyd December 23, 2019 13:27

WillAyd requested changes Dec 23, 2019

View reviewed changes

ShaharNaveh force-pushed the TYP-io-formats-css/csvs branch 2 times, most recently from 56fd651 to 115d3ef Compare December 24, 2019 12:41

ShaharNaveh requested a review from WillAyd December 24, 2019 12:43

WillAyd reviewed Dec 24, 2019

View reviewed changes

ShaharNaveh mentioned this pull request Dec 24, 2019

Type annotations are inconsistent for 'columns' in IO methods #30449

Open

TYP: Typing hints in pandas/io/formats/{css,csvs}.py

3ee967a

ShaharNaveh force-pushed the TYP-io-formats-css/csvs branch from 115d3ef to 6167f1a Compare December 25, 2019 02:31

MomIsBestFriend added 2 commits December 25, 2019 04:31

Added WillAyd's typing suggestions

6167f1a

Review Fixes

1fcab9c

ShaharNaveh requested a review from WillAyd December 25, 2019 19:34

WillAyd added this to the 1.0 milestone Dec 26, 2019

WillAyd approved these changes Dec 26, 2019

View reviewed changes

simonjayhawkins reviewed Dec 26, 2019

View reviewed changes

jreback merged commit 980d0da into pandas-dev:master Dec 27, 2019

ShaharNaveh deleted the TYP-io-formats-css/csvs branch December 28, 2019 17:29

AlexKirko pushed a commit to AlexKirko/pandas that referenced this pull request Dec 29, 2019

TYP: Typing hints in pandas/io/formats/{css,csvs}.py (pandas-dev#30398)

8e90836

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TYP: Typing hints in pandas/io/formats/{css,csvs}.py #30398

TYP: Typing hints in pandas/io/formats/{css,csvs}.py #30398

ShaharNaveh commented Dec 22, 2019 •

edited

WillAyd Dec 23, 2019

ShaharNaveh Dec 24, 2019 •

edited

WillAyd Dec 24, 2019

WillAyd Dec 23, 2019

ShaharNaveh Dec 24, 2019 •

edited

WillAyd Dec 24, 2019

WillAyd Dec 23, 2019

ShaharNaveh Dec 24, 2019 •

edited

WillAyd Dec 24, 2019

WillAyd Dec 23, 2019

ShaharNaveh Dec 24, 2019 •

edited

WillAyd Dec 24, 2019

WillAyd left a comment

WillAyd Dec 24, 2019

WillAyd Dec 24, 2019

WillAyd Dec 24, 2019

WillAyd Dec 24, 2019

WillAyd commented Dec 26, 2019

simonjayhawkins Dec 26, 2019

simonjayhawkins Dec 26, 2019

WillAyd Dec 26, 2019

simonjayhawkins Dec 26, 2019

jreback commented Dec 27, 2019

	line_terminator: Optional[str] = "\n",
	line_terminator: str = "\n",

	compression: Union[str, Mapping[str, str], None] = "infer",
	compression: Optional[Union[str, Mapping[str, str]]] = "infer",

TYP: Typing hints in pandas/io/formats/{css,csvs}.py #30398

TYP: Typing hints in pandas/io/formats/{css,csvs}.py #30398

Conversation

ShaharNaveh commented Dec 22, 2019 • edited

Choose a reason for hiding this comment

ShaharNaveh Dec 24, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShaharNaveh Dec 24, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShaharNaveh Dec 24, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShaharNaveh Dec 24, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd commented Dec 26, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Dec 27, 2019

ShaharNaveh commented Dec 22, 2019 •

edited

ShaharNaveh Dec 24, 2019 •

edited

ShaharNaveh Dec 24, 2019 •

edited

ShaharNaveh Dec 24, 2019 •

edited

ShaharNaveh Dec 24, 2019 •

edited