BUG: Assigning dictionary to series using .loc produces random results.

[rhug123]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame({'A': [3, 9, 3, 4], 'B': [7, 1, 6, 0], 'C': [9, 0, 3, 4], 'D': [1, 8, 0, 0]})

   A  B  C  D
0  3  7  9  1
1  9  1  0  8
2  3  6  3  0
3  4  0  4  0

d = {0:10,1:20,2:30,3:40}

df.loc[:,'A'] = d

Output:

   A  B  C  D
0  0  7  9  1
1  1  1  0  8
2  2  6  3  0
3  3  0  4  0

Issue Description

The values that are assigned to columns A are the keys of the dictionary instead of the values.

If however, instead of assigning the dictionary to an existing column, if we create a new column, we will get the same result the first time we run it, but running the same code again will get the expected result. We then are able to select any column and it will return the expected output.

First time running df.loc[:,'E'] = {0:10,1:20,2:30,3:40}

Output:

   A  B  C  D  E
0  0  7  9  1  0
1  1  1  0  8  1
2  2  6  3  0  2
3  3  0  4  0  3

Second time running df.loc[:,'E'] = {0:10,1:20,2:30,3:40}

   A  B  C  D   E
0  0  7  9  1  10
1  1  1  0  8  20
2  2  6  3  0  30
3  3  0  4  0  40

Then if we run the same code as we did at first, we get a different result:

df.loc[:,'A'] = {0:10,1:20,2:30,3:40}

Output:

    A  B  C  D   E
0  10  7  9  1  10
1  20  1  0  8  20
2  30  6  3  0  30
3  40  0  4  0  40

Expected Behavior

d = {0:10,1:20,2:30,3:40}

df.loc[:,'A'] = d

Output:

     A  B  C  D
 0  10  7  9  1
 1  20  1  0  8
 2  30  6  3  0
 3  40  0  4  0

Installed Versions

1.4.2

[rhshadrach]

Thanks for the report @rhug123. Is there any documentation suggesting assignment using dictionaries like this is supported? In general I believe setting assumes a scalar or list-like. I took a look but couldn't find any, and in general it seems there isn't much documentation on setting with loc.

I'm going to mark this as an enhancement for now until there is some indication it should be supported, in which case it can be changed to a bug.

[rhug123]

Hello @rhshadrach, thank you for taking the time to look at this.

The documentation states under "indexing and selecting data - attribute access (https://pandas.pydata.org/docs/user_guide/indexing.html#attribute-access):

"You can also assign a dict to a row of a DataFrame:"

x = pd.DataFrame({'x': [1, 2, 3], 'y': [3, 4, 5]})

x.iloc[1] = {'x': 9, 'y': 99}

x
Out[26]:
_ x y
0 1 3
1 9 99
2 3 5

While I understand that in the potential bug I have submitted I am looking at assigning a dictionary to a column, as the documentation is referring to a row, I was more concerned with the different behaviors that were observed when running the same code.

[rhshadrach]

Thanks for the link to the documentation; I do think that updating a row does not imply that you can update a column in necessarily similar fashion. While it is tempting to think of "columns are just rows after a transpose", there is an inherent asymmetry with how pandas stores and works with rows vs columns (e.g. dtypes).

I was more concerned with the different behaviors that were observed when running the same code.

This makes sense, and I'm guessing it's due to the block not being consolidated, but if you're using an API in ways that are unsupported then all bets are off. I think a discussion needs to happen on whether the op here should be supported.

[phofl]

The point with assigning dicts to rows is that the keys are matched with the column names. I am not sure if this makes much sense for columns, because you could simply use a Series

[phofl]

Had a chance to take a look. This is a bug, because aligning the dict for the column case was simply forgotten, additionally, the multi-block case already works:

df = pd.DataFrame({"a": [1, 2], "b": [3, 4], "d": "a"})
df.loc[:, "a"] = {1: 100, 0: 200}
df.loc[:, "c"] = {0: 5, 1: 6}

returns

     a  b  d  c
0  200  3  a  0
1  100  4  a  1