Constructing DataFrame from dict

[bmu]

If think it was possible to construct a DataFrame from a dict, which gives an error with v0.15

In [4]: td = {'one': 1, 'two': 2}

In [5]: pd.DataFrame(td)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-71317f07d925> in <module>()
----> 1 pd.DataFrame(td)

.../python2.7/site-packages/pandas/core/frame.pyc in __init__(self, data, index, columns, dtype, copy)
    206                                  dtype=dtype, copy=copy)
    207         elif isinstance(data, dict):
--> 208             mgr = self._init_dict(data, index, columns, dtype=dtype)
    209         elif isinstance(data, ma.MaskedArray):
    210             import numpy.ma.mrecords as mrecords

.../lib/python2.7/site-packages/pandas/core/frame.pyc in _init_dict(self, data, index, columns, dtype)
    334 
    335         return _arrays_to_mgr(arrays, data_names, index, columns,
--> 336                               dtype=dtype)
    337 
    338     def _init_ndarray(self, values, index, columns, dtype=None,

.../python2.7/site-packages/pandas/core/frame.pyc in _arrays_to_mgr(arrays, arr_names, index, columns, dtype)
   4615     # figure out the index, if necessary
   4616     if index is None:
-> 4617         index = extract_index(arrays)
   4618     else:
   4619         index = _ensure_index(index)

.../python2.7/site-packages/pandas/core/frame.pyc in extract_index(data)
   4654 
   4655         if not indexes and not raw_lengths:
-> 4656             raise ValueError('If using all scalar values, you must pass'
   4657                              ' an index')
   4658 

ValueError: If using all scalar values, you must pass an index

The same is true if I use DataFrame.from_dict to construct the DataFrame.

Not sure if this was changed or if it is a bug.

[jorisvandenbossche]

At least back to pandas 0.12 this already gives this same error.

If you want to construct a DataFrame from a simple dict, you should have the items as lists, or else provide an index, or convert it a series:

In [3]: pd.DataFrame({'one': [1], 'two': [2]})
Out[3]:
   one  two
0    1    2

In [4]: pd.DataFrame({'one': 1, 'two': 2}, index=[0])
Out[4]:
   one  two
0    1    2

In [5]: pd.Series({'one': 1, 'two': 2})
Out[5]:
one    1
two    2
dtype: int64

[brazilbean]

The documentation should at least be clarified (although I think this is a bug). When using from_dict, there is no way to pass an index, yet the very name of the method suggests that you should be able to pass a dict and get a DataFrame.

Preferably, from_dict makes an index for you. Otherwise, the documentation needs to clearly state the kinds of dicts that are acceptable.

[brazilbean]

I just discovered this is probably just a misdirected error message in some cases. I get the error when using 2D arrays in my dict:

foo = {'a':random.rand(10,1), 'b':random.rand(10,1)}
pd.DataFrame(foo)

This raises the error stating: "ValueError: If using all scalar values, you must pass an index"

However, if I flatten these arrays, everything works as expected:

foo = {'a':random.rand(10,1).flatten(), 'b':random.rand(10,1).flatten()}
pd.DataFrame(foo)

So, I think the error checking could be improved by distinguishing between scalar-only values and ndarray values.

Keep up the good work!

[jreback]

@brazilbean you are passing a 2-d array to a 1-d object, that's why you get the error. I suppose in the error handler it could maybe interrogate the input a bit more, and generate a better error message. However, that itself could possibly fail (and that's why we don't do this type of thing too much). But if you'd like to submit a PR for that would consider.

[madarasw]

Scenario 1

age_dict = {'Cat': [3], 'Dog': [4], 'Bird': [7]}
age_dataframe = pandas.DataFrame(age_dict)

Scenario 2

age_dict = {'Cat': 3, 'Dog': [4], 'Bird': 7}
age_dataframe = pandas.DataFrame(age_dict)

Scenario 3

age_dict = {'Cat': 3, 'Dog': 4, 'Bird': 7}
age_dataframe = pandas.DataFrame(age_dict)
Considering above three scenarios,
The first scenario gives no error.
The second scenario gives no error when there is at least one list type value in the dictionary, which gives the impression that the third scenario would work.

However, the third scenario gives an error as below;
ValueError: If using all scalar values, you must pass an index