BUG: DataFrame.melt gives unexpected result when column "value" already exists #35003

pizzathief · 2020-06-26T03:30:25Z

[ x] closes BUG: DataFrame.melt gives unexpected result when column "value" already exists #34731
[x ] tests added / passed
[ X] passes black pandas
[ x] passes git diff upstream/master -u -- "*.py" | flake8 --diff
[ x] whatsnew entry

get all the things

pizzathief · 2020-06-26T05:25:44Z

Linting .py code
16
##[error]./pandas/core/apply.py:9:1:F401:'pandas._libs.reduction as libreduction' imported but unused

should I remove this line as part of this PR? "libreduction" isn't mentioned anywhere else in the file pandas/core/apply.py , but on the other hand, its out of scope

WillAyd · 2020-06-26T15:35:34Z

Can you merge master to fix CI issue?

dsaxton · 2020-06-27T00:23:04Z

pandas/tests/reshape/test_melt.py

+        # a name in the dataframe already (default name is "value")
+        df = pd.DataFrame({"col": list("ABC"), "value": range(10, 16, 2)})
+
+        with warnings.catch_warnings(record=True) as w:


Can you use tm.assert_produces_warning here (e.g., as in L580 of this file)?

…nto issue34731-1

a use of a warning detection. Warning.

WillAyd · 2020-06-29T21:19:55Z

pandas/core/reshape/melt.py

@@ -40,6 +41,14 @@ def melt(
    else:
        cols = list(frame.columns)

+    if value_name in frame.columns:
+        warnings.warn(
+            'The value column name "%s" conflicts with an existing '


Can you use f-string syntax? We don't use the Py2 syntax any more

WillAyd · 2020-06-29T21:22:22Z

pandas/core/reshape/melt.py

+        warnings.warn(
+            'The value column name "%s" conflicts with an existing '
+            "column in the dataframe." % (value_name),
+            DeprecationWarning,


Can you change this to a FutureWarning?

WillAyd · 2020-06-29T21:22:51Z

doc/source/whatsnew/v1.1.0.rst

@@ -1105,6 +1105,7 @@ Reshaping
 - Fixed bug in :func:`melt` where melting MultiIndex columns with ``col_level`` > 0 would raise a ``KeyError`` on ``id_vars`` (:issue:`34129`)
 - Bug in :meth:`Series.where` with an empty Series and empty ``cond`` having non-bool dtype (:issue:`34592`)
 - Fixed regression where :meth:`DataFrame.apply` would raise ``ValueError`` for elements whth ``S`` dtype (:issue:`34529`)
+- Issue warning if value column name already exists when using :meth:`DataFrame.melt` (:issue:`34731`)


Can you move this to the Deprecations section?

WillAyd · 2020-06-29T21:23:18Z

pandas/core/reshape/melt.py

+    if value_name in frame.columns:
+        warnings.warn(
+            'The value column name "%s" conflicts with an existing '
+            "column in the dataframe." % (value_name),


Generally we want to give users insight as to what they need to do before the deprecated behavior gets changed - can you try to clarify the message a bit?

jreback · 2020-06-30T12:56:17Z

doc/source/whatsnew/v1.1.0.rst

@@ -816,6 +816,7 @@ Deprecations
 - :meth:`util.testing.assert_almost_equal` now accepts both relative and absolute
  precision through the ``rtol``, and ``atol`` parameters, thus deprecating the
  ``check_less_precise`` parameter. (:issue:`13357`).
+- Issue warning if value column name already exists when using :meth:`DataFrame.melt` (:issue:`34731`)


can you start with DataFrame.melt (like the other notes above), and also say that this is deprecated and removed in future version.

jreback · 2020-06-30T12:56:42Z

pandas/core/reshape/melt.py

@@ -40,6 +41,16 @@ def melt(
    else:
        cols = list(frame.columns)

+    if value_name in frame.columns:
+        warnings.warn(
+            "This dataframe has a column name that matches the value column "


put value in single quotes as this is an argument that the user passed. (and this should be value_name)

jreback · 2020-06-30T12:57:07Z

pandas/core/reshape/melt.py

+    if value_name in frame.columns:
+        warnings.warn(
+            "This dataframe has a column name that matches the value column "
+            f'name of the resultant melted dataframe (That being "{value_name})". '


resultant -> resuling DataFrame. you don't need the parenthesized expression.

jreback · 2020-06-30T12:57:21Z

pandas/core/reshape/melt.py

+        warnings.warn(
+            "This dataframe has a column name that matches the value column "
+            f'name of the resultant melted dataframe (That being "{value_name})". '
+            "In the future this will raise an error, please set the value_name "


value_name in single quotes

jreback · 2020-06-30T12:58:07Z

pandas/tests/reshape/test_melt.py

+        df = pd.DataFrame({"col": list("ABC"), "value": range(10, 16, 2)})
+
+        with tm.assert_produces_warning(FutureWarning):
+            dfm = df.melt(id_vars="value")  # noqa F841


can you create an expected value and use assert_frame_equal

also instead of dfm, call this result.

jreback · 2020-07-02T15:26:48Z

thanks @pizzathief

jbrockmendel · 2022-10-29T01:44:52Z

@pizzathief want to make a PR to enforce this deprecation?

pizzathief added 8 commits June 29, 2018 16:40

Merge pull request #1 from pandas-dev/master

5a84593

get all the things

Merge remote-tracking branch 'upstream/master'

c00363e

warn if value column name already exists

d243026

enblackenated my patch

968fc5b

added entry in whatsnew file

12bdaf7

flake8ified my patch

f20e90f

enacted isortification on patch

ff84c46

Merge remote-tracking branch 'upstream/master'

7267b59

jreback changed the title ~~Issue34731 1~~ BUG: DataFrame.melt gives unexpected result when column "value" already exists Jun 26, 2020

dsaxton reviewed Jun 27, 2020

View reviewed changes

pizzathief added 15 commits June 27, 2020 11:42

Merge remote-tracking branch 'upstream/master'

98acc00

warn if value column name already exists

4a52ccd

enblackenated my patch

e8821d3

added entry in whatsnew file

c7dd8de

flake8ified my patch

0dc55f3

enacted isortification on patch

4b8cbc0

warn if value column name already exists

720713d

enblackenated my patch

f68e322

flake8ified my patch

77003a9

enacted isortification on patch

c9efcee

replaced pytest.warns with assert_produces_warning as requested

e120f23

had to set stacklevel

a9da61e

Merge branch 'issue34731-1' of https://github.com/pizzathief/pandas i…

6a7ad2e

…nto issue34731-1

Gwqot a warning about not removing warnings after I removed

56d2acc

a use of a warning detection. Warning.

reenblackenatified file.

9ff17c4

WillAyd requested changes Jun 29, 2020

View reviewed changes

WillAyd added the Deprecate Functionality to remove in pandas label Jun 29, 2020

requested changes

2d55f27

black says "no" to single quote strings.

d118a90

jreback added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Jun 30, 2020

jreback added this to the 1.1 milestone Jun 30, 2020

jorisvandenbossche mentioned this pull request Jun 30, 2020

DEPR: log of deprecations in 1.x (to be removed in 2.0) #30228

Closed

jreback requested changes Jun 30, 2020

View reviewed changes

pizzathief added 3 commits July 1, 2020 22:21

made further requested changes.

ffaa00c

E231 formatting problems fixed

d2b9fdd

ok black, you can have the comma.

b8d79f4

jreback approved these changes Jul 2, 2020

View reviewed changes

jreback merged commit 997c914 into pandas-dev:master Jul 2, 2020

mroeschke mentioned this pull request Nov 2, 2022

DEPR: Enforce melt(value_name) behavior #49462

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: DataFrame.melt gives unexpected result when column "value" already exists #35003

BUG: DataFrame.melt gives unexpected result when column "value" already exists #35003

pizzathief commented Jun 26, 2020

pizzathief commented Jun 26, 2020

WillAyd commented Jun 26, 2020

dsaxton Jun 27, 2020

WillAyd Jun 29, 2020

WillAyd Jun 29, 2020

WillAyd Jun 29, 2020

WillAyd Jun 29, 2020

jreback Jun 30, 2020

jreback Jun 30, 2020

jreback Jun 30, 2020

jreback Jun 30, 2020

jreback Jun 30, 2020

jreback commented Jul 2, 2020

jbrockmendel commented Oct 29, 2022

BUG: DataFrame.melt gives unexpected result when column "value" already exists #35003

BUG: DataFrame.melt gives unexpected result when column "value" already exists #35003

Conversation

pizzathief commented Jun 26, 2020

pizzathief commented Jun 26, 2020

WillAyd commented Jun 26, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jul 2, 2020

jbrockmendel commented Oct 29, 2022