Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DEPR: Index.to_native_types #36418

Merged
merged 2 commits into from
Sep 19, 2020
Merged

DEPR: Index.to_native_types #36418

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c1e56a9
DEPR: Index.to_native_types
jorisvandenbossche Sep 17, 2020
7381449
fix tests
jorisvandenbossche Sep 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.2.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@ Deprecations
- Deprecated parameter ``dtype`` in :~meth:`Index.copy` on method all index classes. Use the :meth:`Index.astype` method instead for changing dtype(:issue:`35853`)
- Date parser functions :func:`~pandas.io.date_converters.parse_date_time`, :func:`~pandas.io.date_converters.parse_date_fields`, :func:`~pandas.io.date_converters.parse_all_fields` and :func:`~pandas.io.date_converters.generic_parser` from ``pandas.io.date_converters`` are deprecated and will be removed in a future version; use :func:`to_datetime` instead (:issue:`35741`)
- :meth:`DataFrame.lookup` is deprecated and will be removed in a future version, use :meth:`DataFrame.melt` and :meth:`DataFrame.loc` instead (:issue:`18682`)
- The :meth:`Index.to_native_types` is deprecated. Use ``.astype(str)`` instead (:issue:`28867`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking at the docstring for to_native_types, .astype(str) does not cover all the options.

@jorisvandenbossche if I am correct, what the to_native_types method does is add some extra formatting options? These could in principle be useful for users as well? #23820 (comment) 😉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this is almost entirely internal.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those can indeed be potentially useful, but some of them are exposed through display options already, if you want to control how your dataframe is formatted.

Further, I think the main thing is that we simply don't want to have this method in this form public ;) And I mainly wanted to give some alternative (for the default arguments), can also leave it out if will only confuse.
There has been discussion about a generic format() method (#17211)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this is almost entirely internal.

API documented https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.Index.to_native_types.html

And I mainly wanted to give some alternative (for the default arguments), can also leave it out if will only confuse.

since public, I think the release note should stay.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I only meant to remove the " Use .astype(str) instead " part. We regularly have methods that are deprecated without direct alternative if we don't think it is worth mentioning.


.. ---------------------------------------------------------------------------

Expand Down
8 changes: 8 additions & 0 deletions pandas/core/indexes/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1017,6 +1017,8 @@ def to_native_types(self, slicer=None, **kwargs):
"""
Format specified values of `self` and return them.

.. deprecated:: 1.2.0

Parameters
----------
slicer : int, array-like
Expand All @@ -1038,6 +1040,12 @@ def to_native_types(self, slicer=None, **kwargs):
numpy.ndarray
Formatted values.
"""
warnings.warn(
"The 'to_native_types' method is deprecated and will be removed in "
"a future version. Use 'astype(str)' instead.",
FutureWarning,
stacklevel=2,
)
values = self
if slicer is not None:
values = values[slicer]
Expand Down
6 changes: 3 additions & 3 deletions pandas/io/formats/csvs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ def _refine_cols(self, cols: Optional[Sequence[Label]]) -> Sequence[Label]:

if cols is not None:
if isinstance(cols, ABCIndexClass):
cols = cols.to_native_types(**self._number_format)
cols = cols._format_native_types(**self._number_format)
else:
cols = list(cols)
self.obj = self.obj.loc[:, cols]
Expand All @@ -163,7 +163,7 @@ def _refine_cols(self, cols: Optional[Sequence[Label]]) -> Sequence[Label]:
# and make sure sure cols is just a list of labels
cols = self.obj.columns
if isinstance(cols, ABCIndexClass):
return cols.to_native_types(**self._number_format)
return cols._format_native_types(**self._number_format)
else:
assert isinstance(cols, Sequence)
return list(cols)
Expand Down Expand Up @@ -347,5 +347,5 @@ def _save_chunk(self, start_i: int, end_i: int) -> None:
for i in range(len(res.items)):
data[i] = res.iget_values(i)

ix = self.data_index.to_native_types(slicer=slicer, **self._number_format)
ix = self.data_index[slicer]._format_native_types(**self._number_format)
libwriters.write_csv_rows(data, ix, self.nlevels, self.cols, self.writer)
34 changes: 23 additions & 11 deletions pandas/tests/indexes/datetimes/test_formats.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,41 +10,53 @@
import pandas._testing as tm


def test_to_native_types():
def test_to_native_types_method_deprecated():
index = pd.date_range(freq="1D", periods=3, start="2017-01-01")

# First, with no arguments.
expected = np.array(["2017-01-01", "2017-01-02", "2017-01-03"], dtype=object)

result = index.to_native_types()
tm.assert_numpy_array_equal(result, expected)
with tm.assert_produces_warning(FutureWarning):
result = index.to_native_types()

# No NaN values, so na_rep has no effect
result = index.to_native_types(na_rep="pandas")
tm.assert_numpy_array_equal(result, expected)

# Make sure slicing works
expected = np.array(["2017-01-01", "2017-01-03"], dtype=object)

result = index.to_native_types([0, 2])
with tm.assert_produces_warning(FutureWarning):
result = index.to_native_types([0, 2])

tm.assert_numpy_array_equal(result, expected)


def test_to_native_types():
index = pd.date_range(freq="1D", periods=3, start="2017-01-01")

# First, with no arguments.
expected = np.array(["2017-01-01", "2017-01-02", "2017-01-03"], dtype=object)

result = index._format_native_types()
tm.assert_numpy_array_equal(result, expected)

# No NaN values, so na_rep has no effect
result = index._format_native_types(na_rep="pandas")
tm.assert_numpy_array_equal(result, expected)

# Make sure date formatting works
expected = np.array(["01-2017-01", "01-2017-02", "01-2017-03"], dtype=object)

result = index.to_native_types(date_format="%m-%Y-%d")
result = index._format_native_types(date_format="%m-%Y-%d")
tm.assert_numpy_array_equal(result, expected)

# NULL object handling should work
index = DatetimeIndex(["2017-01-01", pd.NaT, "2017-01-03"])
expected = np.array(["2017-01-01", "NaT", "2017-01-03"], dtype=object)

result = index.to_native_types()
result = index._format_native_types()
tm.assert_numpy_array_equal(result, expected)

expected = np.array(["2017-01-01", "pandas", "2017-01-03"], dtype=object)

result = index.to_native_types(na_rep="pandas")
result = index._format_native_types(na_rep="pandas")
tm.assert_numpy_array_equal(result, expected)


Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/indexes/interval/test_formats.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,6 @@ def test_repr_missing(self, constructor, expected):
def test_to_native_types(self, tuples, closed, expected_data):
# GH 28210
index = IntervalIndex.from_tuples(tuples, closed=closed)
result = index.to_native_types()
result = index._format_native_types()
expected = np.array(expected_data)
tm.assert_numpy_array_equal(result, expected)
16 changes: 5 additions & 11 deletions pandas/tests/indexes/period/test_formats.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,35 +12,29 @@ def test_to_native_types():
# First, with no arguments.
expected = np.array(["2017-01-01", "2017-01-02", "2017-01-03"], dtype="=U10")

result = index.to_native_types()
result = index._format_native_types()
tm.assert_numpy_array_equal(result, expected)

# No NaN values, so na_rep has no effect
result = index.to_native_types(na_rep="pandas")
tm.assert_numpy_array_equal(result, expected)

# Make sure slicing works
expected = np.array(["2017-01-01", "2017-01-03"], dtype="=U10")

result = index.to_native_types([0, 2])
result = index._format_native_types(na_rep="pandas")
tm.assert_numpy_array_equal(result, expected)

# Make sure date formatting works
expected = np.array(["01-2017-01", "01-2017-02", "01-2017-03"], dtype="=U10")

result = index.to_native_types(date_format="%m-%Y-%d")
result = index._format_native_types(date_format="%m-%Y-%d")
tm.assert_numpy_array_equal(result, expected)

# NULL object handling should work
index = PeriodIndex(["2017-01-01", pd.NaT, "2017-01-03"], freq="D")
expected = np.array(["2017-01-01", "NaT", "2017-01-03"], dtype=object)

result = index.to_native_types()
result = index._format_native_types()
tm.assert_numpy_array_equal(result, expected)

expected = np.array(["2017-01-01", "pandas", "2017-01-03"], dtype=object)

result = index.to_native_types(na_rep="pandas")
result = index._format_native_types(na_rep="pandas")
tm.assert_numpy_array_equal(result, expected)


Expand Down