Slightly improve DataTree repr

[shoyer]

The DataTree header is now <xarray.DataTree> or <xarray.DataTree 'name'> for consistency with the Dataset and DataArray reprs.

The full path is now included for each DataTree node, rather than printing name and parent separately.

from xarray.core.datatree import DataTree
import xarray as xr

tree = DataTree.from_dict(
    {
        "/": xr.Dataset({"e": (("x",), [1, 2])}, coords={"x": [2, 3]}),
        "/b": xr.Dataset({"f": (("y",), [3])}),
        "/b/c": xr.Dataset(),
        "/b/d": xr.Dataset({"g": 4}),
    }
)

Before:

DataTree('None', parent=None)
│   Dimensions:  (x: 2)
│   Coordinates:
│     * x        (x) int64 16B 2 3
│   Data variables:
│       e        (x) int64 16B 1 2
└── DataTree('b')
    │   Dimensions:  (y: 1)
    │   Dimensions without coordinates: y
    │   Data variables:
    │       f        (y) int64 8B 3
    ├── DataTree('c')
    └── DataTree('d')
            Dimensions:  ()
            Data variables:
                g        int64 8B 4

After:

<xarray.DataTree>
Group: /
│   Dimensions:  (x: 2)
│   Coordinates:
│     * x        (x) int64 16B 2 3
│   Data variables:
│       e        (x) int64 16B 1 2
└── Group: /b
    │   Dimensions:  (y: 1)
    │   Dimensions without coordinates: y
    │   Data variables:
    │       f        (y) int64 8B 3
    ├── Group: /b/c
    └── Group: /b/d
            Dimensions:  ()
            Data variables:
                g        int64 8B 4

[shoyer]

This is ready for another review, after switching the repr of each node from DataTree('name') to DataTree: /path/to/name.

There are two alternatives that are worth considering:

1. Only include name instead of the full path, e.g.,

<xarray.DataTree>
│   Dimensions:  (x: 2)
│   Coordinates:
│     * x        (x) int64 16B 2 3
│   Data variables:
│       e        (x) int64 16B 1 2
└── DataTree: b
    │   Dimensions:  (y: 1)
    │   Dimensions without coordinates: y
    │   Data variables:
    │       f        (y) int64 8B 3
    ├── DataTree: c
    └── DataTree: d
            Dimensions:  ()
            Data variables:
                g        int64 8B 4

I think this is a little less readable, but maybe is a good idea if we think people will mostly be using very heavily nested trees.

2. Use Group: in the repr instead of DataTree:, e.g.,

<xarray.DataTree>
Group: /
│   Dimensions:  (x: 2)
│   Coordinates:
│     * x        (x) int64 16B 2 3
│   Data variables:
│       e        (x) int64 16B 1 2
└── Group: /b
    │   Dimensions:  (y: 1)
    │   Dimensions without coordinates: y
    │   Data variables:
    │       f        (y) int64 8B 3
    ├── Group: /b/c
    └── Group: /b/d
            Dimensions:  ()
            Data variables:
                g        int64 8B 4

I Iike the look of Group a little better than DataTree, so I would lean toward making this change. In fact, this already matches what we have in the HTML repr.

[TomNicholas]

With this change there is now no information about the parent of the top node in the repr. This does matter - it affects what .root will point to, for example. But I could never think of a non-awkward way to include that information, which is why the top node had parent=...?

Only include name instead of the full path, e.g.,

This seems okay.

Use Group: in the repr instead of DataTree:, e.g.,

I tried to avoid using the word "Group" because now we have 3 words for the same thing: "node", "group", and arguably DataTree (since the structure is recursive).

Displaying the full path in each node might quickly get unweildy - e.g. /model_run/scenario/atmospheric_forcing/...

[shoyer]

With this change there is now no information about the parent of the top node in the repr. This does matter - it affects what .root will point to, for example. But I could never think of a non-awkward way to include that information, which is why the top node had parent=...?

This is one reason why I added the full path -- it shows the parent in an integrated part of the repr. But if we go for only printing group names, we could also add a line showing only full path only for the root.

I tried to avoid using the word "Group" because now we have 3 words for the same thing: "node", "group", and arguably DataTree (since the structure is recursive).

I think "DataTree" and "Group" are separate concepts. DataTree is Xarray's name for a tree of groups, coordinates and data variables, in the same way that Dataset is a Xarray's name an aligned collection of coordinates and data variables, and DataArray is Xarray's name for a labeled variable. To give an analogy, "DataTree" is to "group" as "file system" is to "directory."

When you access a coordinate from a DataArray you get another DataArray, but we don't use that name in the repr. It's true that our implementation of DataTree stores nested DataTree objects (unlike the case for DataArray coordinates), but that's an implementation detail that should be opaque to users.

[TomNicholas]

we could also add a line showing only full path only for the root.

I like the idea of having the full path to the root visible somewhere. It's more useful than just parent.

I think "DataTree" and "Group" are separate concepts.

Okay I'm convinced! 😁 I do wonder then whether we should try to replace occurrences of "node" with "group" in the docs...

[shoyer]

Let's ask in the new DataTree meeting for opinions on only showing the name vs. the full path on each group.

[eschalkargans]

Regarding displaying the group name vs full group name, here is an example:

relative

    conditions
        geometry
        mask
            detector_footprint
                r10m
                r20m
                r60m
            l1c_classification
                r60m
        meteorology
            cams
            ecmwf
    measurements
        reflectance
            r10m
            r20m
            r60m
    quality
        mask
            r10m
            r20m
            r60m

absolute

/
    /conditions
        /conditions/geometry
        /conditions/mask
            /conditions/mask/detector_footprint
                /conditions/mask/detector_footprint/r10m
                /conditions/mask/detector_footprint/r20m
                /conditions/mask/detector_footprint/r60m
            /conditions/mask/l1c_classification
                /conditions/mask/l1c_classification/r60m
        /conditions/meteorology
            /conditions/meteorology/cams
            /conditions/meteorology/ecmwf
    /measurements
        /measurements/reflectance
            /measurements/reflectance/r10m
            /measurements/reflectance/r20m
            /measurements/reflectance/r60m
    /quality
        /quality/mask
            /quality/mask/r10m
            /quality/mask/r20m
            /quality/mask/r60m

Regarding my specific case, I don't have any opinion really on the default. However, users with long paths might be annoyed by the absolute paths for sure. But having the full complete path, copy-pastable has probably more value in the everyday life.

[owenlittlejohns]

Looks great. Quick question, probably also to @TomNicholas: is the term "Group" the word we are likely to settle on terminology-wise? (There was discussion of settling on one term for particular concepts in our call this week) I'm not sure if we wanted to decide that now, or if it can wait?

Also noting that we agreed in the same call on the full path being listed for each group/node/subtree, as this PR does.

[shoyer]

I think "group" is the right terminology for the data in a node and its children. It's the same name already used by netCDF, HDF5 and Zarr.

@TomNicholas can you give this a formal review in GitHub? I'd love to get this merged.

[TomNicholas]

Looks great @shoyer. Only thing you could add is a test that checks that the group paths work properly in the case that the node you're printing is not the root of the tree (also a whatsnew entry 😉).

There is a .groups method which returns the whole path, which is consistent with the usage of the term in the repr here.

[shoyer]

Only thing you could add is a test that checks that the group paths work properly in the case that the node you're printing is not the root of the tree

There's a test for this already inside test_repr

(also a whatsnew entry 😉).

I'm going to skip this. DataTree is not a released feature so I don't think users need to know about updates to it.