Mixed datatypes Dataset.to_netcdf()

[mengaldo]

Not sure if this is intended, but when I try to save a Dataset to netCDF using Dataset.to_netcdf(), it produces the following error: ValueError: could not convert string to float: 'a'

This seems to be due to forcing datatypes to be equal to the datatype of the first column encountered. See below a simple working example.

MCVE Code Sample

import pandas as pd
import xarray as xr

df = pd.DataFrame()
df['Values' ] = [1.0,2.0,3.0]
df['Strings'] = ['a','b','c']

da = xr.DataArray(df)
ds = xr.Dataset()
ds['A'] = da
ds.to_netcdf('A.nc')

Expected Output

Gianmarcos-MacBook-Pro-2:examples gian$ python3 test_xarray.py
Traceback (most recent call last):
File "test_xarray.py", line 11, in
ds.to_netcdf('A.nc')
File "/usr/local/lib/python3.7/site-packages/xarray/core/dataset.py", line 1232, in to_netcdf
compute=compute)
File "/usr/local/lib/python3.7/site-packages/xarray/backends/api.py", line 749, in to_netcdf
unlimited_dims=unlimited_dims)
File "/usr/local/lib/python3.7/site-packages/xarray/backends/api.py", line 792, in dump_to_store
unlimited_dims=unlimited_dims)
File "/usr/local/lib/python3.7/site-packages/xarray/backends/common.py", line 261, in store
variables, attributes = self.encode(variables, attributes)
File "/usr/local/lib/python3.7/site-packages/xarray/backends/common.py", line 347, in encode
variables, attributes = cf_encoder(variables, attributes)
File "/usr/local/lib/python3.7/site-packages/xarray/conventions.py", line 605, in cf_encoder
for k, v in iteritems(variables))
File "/usr/local/lib/python3.7/site-packages/xarray/conventions.py", line 605, in
for k, v in iteritems(variables))
File "/usr/local/lib/python3.7/site-packages/xarray/conventions.py", line 241, in encode_cf_variable
var = ensure_dtype_not_object(var, name=name)
File "/usr/local/lib/python3.7/site-packages/xarray/conventions.py", line 201, in ensure_dtype_not_object
data = _copy_with_dtype(data, dtype=_infer_dtype(data, name))
File "/usr/local/lib/python3.7/site-packages/xarray/conventions.py", line 160, in _copy_with_dtype
result[...] = data
ValueError: could not convert string to float: 'a'

Problem Description

It should be able to save the file into netcdf format?

Versions

Output of `xr.show_versions()`

INSTALLED VERSIONS

commit: None
python: 3.7.6 (default, Jan 7 2020, 19:34:32)
[Clang 11.0.0 (clang-1100.0.33.16)]
python-bits: 64
OS: Darwin
OS-release: 19.2.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: C
LOCALE: None.UTF-8
libhdf5: 1.10.2
libnetcdf: 4.4.1.1

xarray: 0.11.2
pandas: 0.24.2
numpy: 1.16.2
scipy: 1.1.0
netCDF4: 1.4.3.2
pydap: None
h5netcdf: None
h5py: 2.8.0
Nio: None
zarr: None
cftime: 1.0.3.4
PseudonetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.19.4
distributed: None
matplotlib: 3.0.2
cartopy: None
seaborn: 0.9.0
setuptools: 42.0.2
pip: 20.0.2
conda: None
pytest: 3.8.2
IPython: 7.1.1
sphinx: 1.8.4

[mathause]

netCDF does not support mixed data types for individual arrays. If you create two arrays, it can be written to disk:

a = df["Values"].to_xarray()
b = df["Strings"].to_xarray()
ds = xr.merge([a, b])
ds.to_netcdf('A.nc')

[max-sixty]

Thanks @mathause .

Closing, please reopen with outstanding questions

[mengaldo]

Thank you both.