DEPR: remove previously-deprecated get_value, set_value #27377

jbrockmendel · 2019-07-13T05:07:10Z

Deprecated in 0.21.0

jreback · 2019-07-13T05:20:48Z

pandas/tests/frame/test_indexing.py

@@ -1974,9 +1963,8 @@ def test_get_set_value_no_partial_indexing(self):
        # partial w/ MultiIndex raise exception
        index = MultiIndex.from_tuples([(0, 1), (0, 2), (1, 1), (1, 2)])
        df = DataFrame(index=index, columns=range(4))


i would blow most/all of the _get_value and _set_value test away

im not sure we have other tests for _get_value/_set_value, which are still used internally

Any thought to migrating it to test_interals then? I can see the value of separating these out from other tests at the very least

jorisvandenbossche · 2019-07-13T14:18:20Z

Note I have nothing against this PR itself, but I don't think we should be removing things after the RC.

jbrockmendel · 2019-07-13T17:02:31Z

that makes sense, will move note to the 0.26.0 file

jbrockmendel · 2019-07-13T17:03:38Z

(or, since there is no 0.26.0 file yet, will wait until there is)

WillAyd

lgtm and indifferent on timing question

WillAyd · 2019-07-15T00:28:50Z

pandas/tests/frame/test_indexing.py

@@ -1974,9 +1963,8 @@ def test_get_set_value_no_partial_indexing(self):
        # partial w/ MultiIndex raise exception
        index = MultiIndex.from_tuples([(0, 1), (0, 2), (1, 1), (1, 2)])
        df = DataFrame(index=index, columns=range(4))


Any thought to migrating it to test_interals then? I can see the value of separating these out from other tests at the very least

…value

jbrockmendel · 2019-07-15T16:42:19Z

Any thought to migrating it to test_interals then? I can see the value of separating these out from other tests at the very least

Looks like these two methods are used in tests scattered across a handful of files. Getting a better organization for indexing tests is worthwhile, but is a pretty big task

…value

jbrockmendel · 2019-07-19T15:02:02Z

created a 0.26.0 file and moved the deprecation-removal there

TomAugspurger · 2019-07-19T16:32:08Z

Are we just pretending like we're doing 0.26.0 for now, and moving to 1.0 if we decide we're ready? I think that makes the most sense.

…

On Fri, Jul 19, 2019 at 10:02 AM jbrockmendel ***@***.***> wrote: created a 0.26.0 file and moved the deprecation-removal there — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#27377?email_source=notifications&email_token=AAKAOIVJJRPM3O7EOVVZAYTQAHJPHA5CNFSM4ICW3Z3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2L4OFQ#issuecomment-513263382>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKAOIUQBC2VDK7XDRTZLTLQAHJPHANCNFSM4ICW3Z3A> .

jbrockmendel · 2019-07-19T16:36:29Z

Are we just pretending like we're doing 0.26.0 for now, and moving to 1.0 if we decide we're ready? I think that makes the most sense.

That's the impression I got from [dont remember, but I'm 80% sure it wasn't a dream]

WillAyd · 2019-07-19T16:39:35Z

Just added the milestone - can change name as required

jreback

so are you sure we cannot simply remove entirely get_value and set_value? where exactly are these used outside of tests?

I see I think only

pandas/core//indexing.py:        self.obj._set_value(*key, takeable=self._takeable)

which could be use iat/at

jreback · 2019-07-20T20:47:40Z

doc/source/whatsnew/v0.26.0.rst

+What's new in 0.26.0 (July XX, 2019)
+------------------------------------
+
+Enhancements


added as 1.0, so this can be removed

jreback · 2019-07-20T20:47:48Z

doc/source/whatsnew/v0.26.0.rst

+
+Removal of prior version deprecations/changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+- Removed the previously deprecated :meth:`Series.get_value`, :meth:`Series.set_value`, :meth:`DataFrame.get_value`, :meth:`DataFrame.set_value` (:issue:`17739`)


move to 1.0

…value

jbrockmendel · 2019-07-20T23:48:37Z

I see I think only [1 usage]

I count 15 non-test usages

jreback · 2019-07-20T23:53:46Z

I see I think only [1 usage]

I count 15 non-test usages

show me the list

jbrockmendel · 2019-07-20T23:57:12Z

Searching 1589 files for "._get_value(|._set_value(" (regex, case sensitive)

pandas/core/frame.py:
2823 col = self.columns.get_loc(col)
2824 index = self.index.get_loc(index)
2825: return self._get_value(index, col, takeable=True)
2826
2827 def _set_value(self, index, col, value, takeable: bool = False):
....
2845 if takeable is True:
2846 series = self._iget_item_cache(col)
2847: return series._set_value(index, value, takeable=True)
2848
2849 series = self._get_item_cache(col)
....
3786 result = np.empty(n, dtype="O")
3787 for i, (r, c) in enumerate(zip(row_labels, col_labels)):
3788: result[i] = self._get_value(r, c)
3789
3790 if is_object_dtype(result):

pandas/core/indexing.py:
123 key = tuple(com.apply_if_callable(x, self.obj) for x in key)
124 try:
125: values = self.obj._get_value(*key)
126 except (KeyError, TypeError, InvalidIndexError):
127 # TypeError occurs here if the key has non-hashable entries,
...
1735 # a fast-path to scalar access
1736 # if not, raise
1737: values = self.obj._get_value(*key)
1738 return values
1739
....
2035 # a fast-path to scalar access
2036 # if not, raise
2037: values = self.obj._get_value(*key, takeable=True)
2038 return values
2039
....
2173
2174 key = self._convert_key(key)
2175: return self.obj._get_value(*key, takeable=self._takeable)
2176
2177 def setitem(self, key, value):
....
2188 key = list(self._convert_key(key, is_setter=True))
2189 key.append(value)
2190: self.obj._set_value(*key, takeable=self._takeable)
2191
2192

pandas/core/sparse/frame.py:
473 stacklevel=2,
474 )
475: return self._get_value(index, col, takeable=takeable)
476
477 def _get_value(self, index, col, takeable=False):
...
481 series = self._get_item_cache(col)
482
483: return series._get_value(index, takeable=takeable)
484
485 _get_value.doc = get_value.doc
...
517 stacklevel=2,
518 )
519: return self._set_value(index, col, value, takeable=takeable)
520
521 def _set_value(self, index, col, value, takeable=False):
522: dense = self.to_dense()._set_value(index, col, value, takeable=takeable)
523 return dense.to_sparse(
524 kind=self._default_kind, fill_value=self._default_fill_value

pandas/core/sparse/series.py:
350
351 def _set_with_engine(self, key, value):
352: return self._set_value(key, value)
353
354 def abs(self):
...
412 )
413
414: return self._get_value(label, takeable=takeable)
415
416 def _get_value(self, label, takeable=False):
...
454 stacklevel=2,
455 )
456: return self._set_value(label, value, takeable=takeable)
457
458 def _set_value(self, label, value, takeable=False):
...
461 # if the label doesn't exist, we will create a new object here
462 # and possibly change the index
463: new_values = values._set_value(label, value, takeable=takeable)
464 if new_values is not None:
465 values = new_values

pandas/tests/frame/test_api.py:
68 for idx in float_frame.index:
69 for col in float_frame.columns:
70: result = float_frame._get_value(idx, col)
71 expected = float_frame[col][idx]
72 tm.assert_almost_equal(result, expected)

pandas/tests/frame/test_constructors.py:
330 frame_none = DataFrame(dict(a=None), index=[0])
331 frame_none_list = DataFrame(dict(a=[None]), index=[0])
332: assert frame_none._get_value(0, "a") is None
333: assert frame_none_list._get_value(0, "a") is None
334 tm.assert_frame_equal(frame_none, frame_none_list)
335
...
701 for col in df.columns:
702 for row in df.index:
703: data.setdefault(col, {})[row] = df._get_value(row, col)
704
705 result = DataFrame(data, columns=rng)
...
709 for col in df.columns:
710 for row in df.index:
711: data.setdefault(row, {})[col] = df._get_value(row, col)
712
713 result = DataFrame(data, index=rng).T

pandas/tests/frame/test_indexing.py:
1850 for idx in float_frame.index:
1851 for col in float_frame.columns:
1852: result = float_frame._get_value(idx, col)
1853 expected = float_frame[col][idx]
1854 assert result == expected
....
1905 for idx in float_frame.index:
1906 for col in float_frame.columns:
1907: float_frame._set_value(idx, col, 1)
1908 assert float_frame[col][idx] == 1
1909
1910 def test_set_value_resize(self, float_frame):
1911
1912: res = float_frame._set_value("foobar", "B", 0)
1913 assert res is float_frame
1914 assert res.index[-1] == "foobar"
1915: assert res._get_value("foobar", "B") == 0
1916
1917 float_frame.loc["foobar", "qux"] = 0
1918: assert float_frame._get_value("foobar", "qux") == 0
1919
1920 res = float_frame.copy()
1921: res3 = res.set_value("foobar", "baz", "sam")
1922 assert res3["baz"].dtype == np.object
1923
1924 res = float_frame.copy()
1925: res3 = res.set_value("foobar", "baz", True)
1926 assert res3["baz"].dtype == np.object
1927
1928 res = float_frame.copy()
1929: res3 = res._set_value("foobar", "baz", 5)
1930 assert is_float_dtype(res3["baz"])
1931 assert isna(res3["baz"].drop(["foobar"])).all()
1932 msg = "could not convert string to float: 'sam'"
1933 with pytest.raises(ValueError, match=msg):
1934: res3._set_value("foobar", "baz", "sam")
1935
1936 def test_set_value_with_index_dtype_change(self):
....
1940 # so column is not created
1941 df = df_orig.copy()
1942: df._set_value("C", 2, 1.0)
1943 assert list(df.index) == list(df_orig.index) + ["C"]
1944 # assert list(df.columns) == list(df_orig.columns) + [2]
....
1951 # create both new
1952 df = df_orig.copy()
1953: df._set_value("C", "D", 1.0)
1954 assert list(df.index) == list(df_orig.index) + ["C"]
1955 assert list(df.columns) == list(df_orig.columns) + ["D"]
....
1965 df = DataFrame(index=index, columns=range(4))
1966 with pytest.raises(KeyError, match=r"^0$"):
1967: df._get_value(0, 1)
1968
1969 def test_single_element_ix_dont_upcast(self, float_frame):

pandas/tests/series/indexing/test_datetime.py:
111 index = DatetimeIndex(dates)
112
113: s = Series()._set_value(dates[0], 1.0)
114: s2 = s._set_value(dates[1], np.nan)
115
116 expected = Series([1.0, np.nan], index=index)

pandas/tests/series/indexing/test_indexing.py:
373 def test_set_value(test_data):
374 idx = test_data.ts.index[10]
375: res = test_data.ts._set_value(idx, 0)
376 assert res is test_data.ts
377 assert test_data.ts[idx] == 0
...
379 # equiv
380 s = test_data.series.copy()
381: res = s._set_value("foobar", 0)
382 assert res is s
383 assert res.index[-1] == "foobar"

37 matches across 9 files

jreback · 2019-07-21T00:01:27Z

@jbrockmendel

ok I guess this is not trivial to remove. However, this does need rationalization (e.g. the name is not inline with any other indexers, even internally that we have). So this should be addressed at some point; these routines likely are not great at handling for example non-basic dtypes as these never got love and have been externally replaced by iat/at.

So if we don't have an issue can you create one.thanks.

jreback · 2019-07-21T00:02:49Z

@jbrockmendel if you can't tell, I really really dislike doing these private version of public methods just so we can deprecate them, rather than actually fix things. This is just a propagation of technical debt and really hurting the project (just a bit of venting, you are certainly addressing lots of this prior techinical debt).

jreback · 2019-07-21T00:02:56Z

ping on green.

jbrockmendel · 2019-07-21T00:11:39Z

Definitely agreed on getting rid of the private versions before long. I've been looking at indexing methods and hoping to get some real simplification done before long.

jbrockmendel · 2019-07-21T20:55:40Z

Ping

…value

jbrockmendel · 2019-07-22T21:00:23Z

Updated to also remove the deprecated methods from SparseSeries, SparseDataFrame.

It looks like iat and at dispatch to _get_value/_set_value. After this I'd like to do a follow-up to move the contents of _get_value/_set_value to iat/at and get rid of _get_value/_set_value altogether.

…value

jreback · 2019-07-23T22:27:14Z

can you rebase again just to make sure here

…value

jbrockmendel · 2019-07-24T00:34:12Z

ping

jreback · 2019-07-24T00:45:02Z

thanks

…7377) * DEPR: remove previously-deprecated get_value, set_value * start 0.26.0 file, move deprecation removal there * whitespace fixup * move note to 1.0 file * also remove for Sparse * remove usages of get_value, set_value

DEPR: remove previously-deprecated get_value, set_value

33c901c

jreback reviewed Jul 13, 2019

View reviewed changes

jorisvandenbossche mentioned this pull request Jul 13, 2019

RLS: 0.25.0 #24950

Closed

simonjayhawkins added Deprecate Functionality to remove in pandas Indexing Related to indexing on series/frames, not to indexes themselves labels Jul 13, 2019

WillAyd approved these changes Jul 15, 2019

View reviewed changes

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm…

3f13796

…value

jbrockmendel added 2 commits July 19, 2019 07:55

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm…

d7aa003

…value

start 0.26.0 file, move deprecation removal there

4411126

WillAyd added this to the 0.26.0 milestone Jul 19, 2019

whitespace fixup

cf90e5d

jreback modified the milestones: 0.26.0, 1.0 Jul 20, 2019

jreback requested changes Jul 20, 2019

View reviewed changes

jbrockmendel added 2 commits July 20, 2019 16:45

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm…

e622fb1

…value

move note to 1.0 file

615bb1d

jreback approved these changes Jul 21, 2019

View reviewed changes

jbrockmendel added 2 commits July 22, 2019 13:49

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm…

7163c77

…value

also remove for Sparse

bfde54c

TomAugspurger approved these changes Jul 22, 2019

View reviewed changes

jbrockmendel added 2 commits July 22, 2019 14:42

remove usages of get_value, set_value

72751b7

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm…

5789b57

…value

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm…

a6dfca9

…value

jreback merged commit 0b02137 into pandas-dev:master Jul 24, 2019

jbrockmendel deleted the rmvalue branch July 24, 2019 00:58

WillAyd mentioned this pull request Jul 26, 2019

Removed get_value benchmarks #27603

Merged

jreback mentioned this pull request Jul 26, 2019

DEPR: deprecations log for removed issues #13777

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPR: remove previously-deprecated get_value, set_value #27377

DEPR: remove previously-deprecated get_value, set_value #27377

jbrockmendel commented Jul 13, 2019

jreback Jul 13, 2019 •

edited

jbrockmendel Jul 13, 2019

WillAyd Jul 15, 2019

jorisvandenbossche commented Jul 13, 2019

jbrockmendel commented Jul 13, 2019

jbrockmendel commented Jul 13, 2019

WillAyd left a comment

WillAyd Jul 15, 2019

jbrockmendel commented Jul 15, 2019

jbrockmendel commented Jul 19, 2019

TomAugspurger commented Jul 19, 2019 via email

jbrockmendel commented Jul 19, 2019

WillAyd commented Jul 19, 2019

jreback left a comment

jreback Jul 20, 2019

jreback Jul 20, 2019

jbrockmendel Jul 20, 2019

jbrockmendel commented Jul 20, 2019

jreback commented Jul 20, 2019

jbrockmendel commented Jul 20, 2019

jreback commented Jul 21, 2019

jreback commented Jul 21, 2019

jreback commented Jul 21, 2019

jbrockmendel commented Jul 21, 2019

jbrockmendel commented Jul 21, 2019

jbrockmendel commented Jul 22, 2019

jreback commented Jul 23, 2019

jbrockmendel commented Jul 24, 2019

jreback commented Jul 24, 2019

DEPR: remove previously-deprecated get_value, set_value #27377

DEPR: remove previously-deprecated get_value, set_value #27377

Conversation

jbrockmendel commented Jul 13, 2019

jreback Jul 13, 2019 • edited

Choose a reason for hiding this comment

jbrockmendel Jul 13, 2019

Choose a reason for hiding this comment

WillAyd Jul 15, 2019

Choose a reason for hiding this comment

jorisvandenbossche commented Jul 13, 2019

jbrockmendel commented Jul 13, 2019

jbrockmendel commented Jul 13, 2019

WillAyd left a comment

Choose a reason for hiding this comment

WillAyd Jul 15, 2019

Choose a reason for hiding this comment

jbrockmendel commented Jul 15, 2019

jbrockmendel commented Jul 19, 2019

TomAugspurger commented Jul 19, 2019 via email

jbrockmendel commented Jul 19, 2019

WillAyd commented Jul 19, 2019

jreback left a comment

Choose a reason for hiding this comment

jreback Jul 20, 2019

Choose a reason for hiding this comment

jreback Jul 20, 2019

Choose a reason for hiding this comment

jbrockmendel Jul 20, 2019

Choose a reason for hiding this comment

jbrockmendel commented Jul 20, 2019

jreback commented Jul 20, 2019

jbrockmendel commented Jul 20, 2019

jreback commented Jul 21, 2019

jreback commented Jul 21, 2019

jreback commented Jul 21, 2019

jbrockmendel commented Jul 21, 2019

jbrockmendel commented Jul 21, 2019

jbrockmendel commented Jul 22, 2019

jreback commented Jul 23, 2019

jbrockmendel commented Jul 24, 2019

jreback commented Jul 24, 2019

jreback Jul 13, 2019 •

edited