Dataset constructor fails if values are objects

[max-sixty]

For an item to be a value in a Dataset, it either needs to be array-like, or pass lib.is_scalar.

I think this is probably too strict - I'd propose anything can be a value, and it only gets treated as an array if it looks like one - thoughts?

Here's the code that checks the values: https://github.com/pydata/xarray/blob/master/xarray/core/variable.py#L52

In [13]: class B(object):
    pass
   ....: 

In [14]: xr.Dataset({'a':B()})
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-14-b2eeb6de19ea> in <module>()
----> 1 xr.Dataset({'a':B()})

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/dataset.py in __init__(self, data_vars, coords, attrs, compat, **kwargs)
    207             coords = set()
    208         if data_vars is not None or coords is not None:
--> 209             self._set_init_vars_and_dims(data_vars, coords, compat)
    210         if attrs is not None:
    211             self.attrs = attrs

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/dataset.py in _set_init_vars_and_dims(self, vars, coords, compat)
    258         aligned = align_variables(variables)
    259         new_variables, new_coord_names = expand_variables(aligned,
--> 260                                                           compat=compat)
    261 
    262         new_coord_names.update(coords)

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/merge.py in expand_variables(raw_variables, old_variables, compat)
     75                     add_variable(dim, coord.variable)
     76             var = var.variable
---> 77         add_variable(name, var)
     78 
     79     return new_variables, new_coord_names

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/merge.py in add_variable(name, var)
     55 
     56     def add_variable(name, var):
---> 57         var = _as_dataset_variable(name, var)
     58         if name not in variables:
     59             variables[name] = var

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/merge.py in _as_dataset_variable(name, var)
      9     """
     10     try:
---> 11         var = as_variable(var, key=name)
     12     except TypeError:
     13         raise TypeError('variables must be given by arrays or a tuple of '

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/variable.py in as_variable(obj, key, strict, copy)
     55             obj = Variable(obj.name, obj)
     56         elif key is not None:
---> 57             obj = Variable(key, obj)
     58         else:
     59             raise TypeError('cannot infer Variable dimensions')

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/variable.py in __init__(self, dims, data, attrs, encoding, fastpath)
    211         """
    212         self._data = as_compatible_data(data, fastpath=fastpath)
--> 213         self._dims = self._parse_dimensions(dims)
    214         self._attrs = None
    215         self._encoding = None

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/variable.py in _parse_dimensions(self, dims)
    319             raise ValueError('dimensions %s must have the same length as the '
    320                              'number of data dimensions, ndim=%s'
--> 321                              % (dims, self.ndim))
    322         return dims
    323 

ValueError: dimensions ('a',) must have the same length as the number of data dimensions, ndim=0

[shoyer]

You can put anything you want in a Dataset if you provide dimensions explicitly:

In [1]: import xarray as xr

In [2]: class Foo:
   ...:     pass
   ...:

In [3]: xr.Dataset({'foo': ([], Foo())})
Out[3]:
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    *empty*
Data variables:
    foo      object <__main__.Foo object at 0x10824cba8>

The problem is that we need some rule to detect the dimensionality of input values, to know if we can treat them as a scalar or if we should raise an error. There are other ways to do this other than checking against a white-list of scalar types, but we can't take the obvious approach of converting everything into a numpy array and then checking the dimensionality because this can't be done safely for some types (e.g., dask.array).

I'm certainly open to ideas on how to improve this. At the very least, we should certainly improve the error message -- the error about mis-matched dimensions arises because we assume that anything that isn't a scalar but that is used as a key in the data_vars dict is a 1D coordinate variable along a dimension.

[max-sixty]

There are other ways to do this other than checking against a white-list of scalar types, but we can't take the obvious approach of converting everything into a numpy array and then checking the dimensionality because this can't be done safely for some types (e.g., dask.array).

What do you think about the reverse approach - checking for a list of known array types, and everything else becomes a scalar?

[shoyer]

What do you think about the reverse approach - checking for a list of known array types, and everything else becomes a scalar?

Sure, we could probably make this work