Skip to content

Commit

Permalink
Explicitly namespace dim expressions (#4320)
Browse files Browse the repository at this point in the history
  • Loading branch information
philippjfr committed Mar 24, 2020
1 parent d5b3532 commit c1894b7
Show file tree
Hide file tree
Showing 5 changed files with 550 additions and 250 deletions.
13 changes: 7 additions & 6 deletions examples/user_guide/11-Transforming_Elements.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
"metadata": {},
"outputs": [],
"source": [
"expr = np.sin(hv.dim('x')*10+5)\n",
"expr = np.sin(hv.dim('x').df*10+5)\n",
"expr"
]
},
Expand Down Expand Up @@ -106,7 +106,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This allows almost any mathematical transformation to be expressed and applied on a `Dataset` in a deferred way. The `dim` expression supports all the standard mathematical operators, NumPy array methods but may also be used to apply methods which exist only on the datatype the data as stored as. Let us for example load an XArray Dataset, which has a number of custom methods to perform complex computations on the data, e.g. the quantile method:"
"This allows almost any mathematical transformation to be expressed and applied on a `Dataset` in a deferred way. The regular `dim` expression supports all the standard mathematical operators and NumPy array methods. However if we want to use methods which exist only on specific datatypes we can invoke them using `.df` or `.xr`, which let you make (pandas) dataframe and xarray API (method and accessor) calls respectively. Let us for example load an XArray Dataset, which has a number of custom methods to perform complex computations on the data, e.g. the quantile method:"
]
},
{
Expand Down Expand Up @@ -134,7 +134,7 @@
"metadata": {},
"outputs": [],
"source": [
"quantile_expr = hv.dim('air').quantile(0.95, dim='time')\n",
"quantile_expr = hv.dim('air').xr.quantile(0.95, dim='time')\n",
"quantile_expr"
]
},
Expand All @@ -151,7 +151,7 @@
"metadata": {},
"outputs": [],
"source": [
"temp_ds = hv.Dataset(air_temp, ['lon', 'lat'])\n",
"temp_ds = hv.Dataset(air_temp, ['lon', 'lat', 'time'])\n",
"\n",
"transformed_ds = temp_ds.transform(air=quantile_expr)\n",
"\n",
Expand Down Expand Up @@ -191,7 +191,7 @@
"\n",
"q = pn.widgets.FloatSlider(name='quantile')\n",
"\n",
"quantile_expr = hv.dim('air').quantile(q, dim='time')\n",
"quantile_expr = hv.dim('air').xr.quantile(q, dim='time')\n",
"quantile_expr"
]
},
Expand All @@ -209,8 +209,9 @@
"outputs": [],
"source": [
"temp_ds = hv.Dataset(air_temp, ['lon', 'lat'])\n",
"transformed = temp_ds.apply.transform(air=quantile_expr).apply(hv.Image)\n",
"\n",
"pn.Column(q, temp_ds.apply.transform(air=quantile_expr).apply(hv.Image).opts(colorbar=True, width=400))"
"pn.Column(q, transformed.opts(colorbar=True, width=400))"
]
},
{
Expand Down
8 changes: 4 additions & 4 deletions examples/user_guide/14-Data_Pipelines.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The ``.apply`` method allows us to automatically build a pipeline given an object and some operation or function along with parameter instances passed in as keyword arguments. Internally it will then build a Stream to ensure that whenever the parameter changes the plot is updated. To learn more about streams see the [Responding to Events](./12-Responding_to_Events.ipynb).\n",
"The ``.apply`` method allows us to automatically build a dynamic pipeline given an object and some operation or function along with parameter, stream or widget instances passed in as keyword arguments. Internally it will then build a `Stream` to ensure that whenever one of these changes the plot is updated. To learn more about streams see the [Responding to Events](./12-Responding_to_Events.ipynb).\n",
"\n",
"This mechanism allows us to build powerful pipelines by linking parameters on a user defined class or even an external widget, e.g. here we import an ``IntSlider`` widget from [``panel``](https://pyviz.panel.org):"
]
Expand All @@ -131,7 +131,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the ``.apply`` method we can now apply the ``rolling`` operation to the DynamicMap and link the slider to the operation's ``rolling_window`` parameter (which also works for simple functions as will be shown below). However to further demonstrate the features of `dim` expressions and the `.transform` method which we first introduced in the [Transforming elements user guide](11-Transforming_Elements.ipynb) we will instead apply the rolling mean using a `dim` expression instead:"
"Using the ``.apply`` method we could now apply the ``rolling`` operation to the DynamicMap and link the slider to the operation's ``rolling_window`` parameter (which also works for simple functions as will be shown below). However, to further demonstrate the features of `dim` expressions and the `.transform` method, which we first introduced in the [Transforming elements user guide](11-Transforming_Elements.ipynb), we will instead apply the rolling mean using the `.df` namespace accessor on a `dim` expression:"
]
},
{
Expand All @@ -140,7 +140,7 @@
"metadata": {},
"outputs": [],
"source": [
"rolled_dmap = dmap.apply.transform(adj_close=hv.dim('adj_close').rolling(slider).mean())\n",
"rolled_dmap = dmap.apply.transform(adj_close=hv.dim('adj_close').df.rolling(slider).mean())\n",
"\n",
"rolled_dmap"
]
Expand Down Expand Up @@ -242,5 +242,5 @@
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 4
}
16 changes: 11 additions & 5 deletions holoviews/core/data/xarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -625,6 +625,7 @@ def assign(cls, dataset, new_data):
coords[k] = (k, v)
if coords:
data = data.assign_coords(**coords)

dims = tuple(kd.name for kd in dataset.kdims[::-1])
vars = OrderedDict()
for k, v in new_data.items():
Expand All @@ -634,11 +635,16 @@ def assign(cls, dataset, new_data):
vars[k] = v
else:
vars[k] = (dims, cls.canonicalize(dataset, v, data_coords=dims))
if vars:
data = data.assign(vars)
used_coords = set.intersection(*[set(var.coords) for var in data.data_vars.values()])
drop_coords = set.symmetric_difference(used_coords, prev_coords)
return data.drop(list(drop_coords)), list(drop_coords)

if len(vars) == 1 and list(vars) == [vd.name for vd in dataset.vdims]:
data = vars[dataset.vdims[0].name]
used_coords = set(data.coords)
else:
if vars:
data = data.assign(vars)
used_coords = set.intersection(*[set(var.coords) for var in data.data_vars.values()])
drop_coords = set.symmetric_difference(used_coords, prev_coords)
return data.drop([c for c in drop_coords if c in data.coords]), list(drop_coords)


Interface.register(XArrayInterface)
Loading

0 comments on commit c1894b7

Please sign in to comment.