Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grid.refine raises incomprehensible ValueError when there is a data variable with dtype object present in the dataset #208

Closed
OnnoEbbens opened this issue Jul 20, 2023 · 0 comments

Comments

@OnnoEbbens
Copy link
Collaborator

When I add the code below after the 4th code cell in the example notebook 03_local_grid_refinement I get the ValueError below. The origin of the error is a problem when using ds.interp to refine the data variables in the dataset. The interp function gives a different result when it has to modify data variables with type object.

I will try to fix this by removing the data variables with type object from the dataset. I think in most cases this won't be a problem. If it is we can try to think of a method to refine arrays of type object. This should be doable for the method 'nearest'.

import numpy as np
ones = np.ones((ds.dims['y'], ds.dims['x']),dtype=str)
ds['davar_str'] = (('y','x'), ones)
ds = nlmod.grid.refine(ds)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[~\AppData\Local\Temp\ipykernel_7768\2380354137.py](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/nlmod/docs/examples/~/AppData/Local/Temp/ipykernel_7768/2380354137.py) in ()
      2 ones = np.ones((ds.dims['y'], ds.dims['x']),dtype=str)
      3 ds['davar_str'] = (('y','x'), ones)
----> 4 ds = nlmod.grid.refine(ds)

[~\02_python\nlmod\nlmod\dims\grid.py](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/nlmod/docs/examples/~/02_python/nlmod/nlmod/dims/grid.py) in refine(ds, model_ws, refinement_features, exe_name, remove_nan_layers, model_coordinates)
    423     gridprops = g.get_gridprops_disv()
    424     gridprops["area"] = g.get_area()
--> 425     ds = ds_to_gridprops(ds, gridprops=gridprops)
    426     # recalculate idomain, as the interpolation changes idomain to floats
    427     ds = set_idomain(ds, remove_nan_layers=remove_nan_layers)

[~\02_python\nlmod\nlmod\dims\grid.py](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/nlmod/docs/examples/~/02_python/nlmod/nlmod/dims/grid.py) in ds_to_gridprops(ds_in, gridprops, method, nodata)
    475         # only keep the first layer of area
    476         area = gridprops["area"][: len(ds_out["icell2d"])]
--> 477         ds_out["area"] = ("icell2d", area)
    478 
    479     # add information about the vertices

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\dataset.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/dataset.py) in __setitem__(self, key, value)
   1495                     "object can be stored under a single key."
   1496                 )
-> 1497             self.update({key: value})
   1498 
   1499         elif utils.iterable_of_hashable(key):

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\dataset.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/dataset.py) in update(self, other)
   4951         Dataset.merge
   4952         """
-> 4953         merge_result = dataset_update_method(self, other)
   4954         return self._replace(inplace=True, **merge_result._asdict())
   4955 

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\merge.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/merge.py) in dataset_update_method(dataset, other)
   1091                     other[key] = value.drop_vars(coord_names)
   1092 
-> 1093     return merge_core(
   1094         [dataset, other],
   1095         priority_arg=1,

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\merge.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/merge.py) in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value)
    739 
    740     coerced = coerce_pandas_values(objects)
--> 741     aligned = deep_align(
    742         coerced, join=join, copy=False, indexes=indexes, fill_value=fill_value
    743     )

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\alignment.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/alignment.py) in deep_align(objects, join, copy, indexes, exclude, raise_on_invalid, fill_value)
    848             out.append(variables)
    849 
--> 850     aligned = align(
    851         *targets,
    852         join=join,

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\alignment.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/alignment.py) in align(join, copy, indexes, exclude, fill_value, *objects)
    785         fill_value=fill_value,
    786     )
--> 787     aligner.align()
    788     return aligner.results
    789 

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\alignment.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/alignment.py) in align(self)
    569         self.find_matching_indexes()
    570         self.find_matching_unindexed_dims()
--> 571         self.assert_no_index_conflict()
    572         self.align_indexes()
    573         self.assert_unindexed_dim_sizes_equal()

[c:\Users\oebbe\anaconda3\envs\nlmod\lib\site-packages\xarray\core\alignment.py](file:///C:/Users/oebbe/anaconda3/envs/nlmod/lib/site-packages/xarray/core/alignment.py) in assert_no_index_conflict(self)
    312                     f"{k!r} ({v} conflicting indexes)" for k, v in dup.items()
    313                 )
--> 314                 raise ValueError(
    315                     "cannot re-index or align objects with conflicting indexes found for "
    316                     f"the following {msg}: {items_msg}\n"

ValueError: cannot re-index or align objects with conflicting indexes found for the following dimensions: 'icell2d' (2 conflicting indexes)
Conflicting indexes may occur when
- they relate to different sets of coordinate and/or dimension names
- they don't have the same type
- they may be used to reindex data along common dimensions

OnnoEbbens added a commit that referenced this issue Jul 20, 2023
OnnoEbbens added a commit that referenced this issue Jul 20, 2023
…a variable with dtype object present in the dataset (#209)

* fix for #208

* better log message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant