Add method to format xarray.DataArray as strings for issue #5985 #7628

husainridwan · 2023-03-15T00:46:06Z

Closes Formatting data array as strings? #5985
Tests added
User visible changes (including notable bug fixes) are documented in whats-new.rst
New functions/methods are listed in api.rst

…into round-stringify

for more information, see https://pre-commit.ci

husainridwan · 2023-03-15T00:48:53Z

@TomNicholas Can you review this for me?

headtr1ck · 2023-03-15T09:43:57Z

Wouldn't it be more useful to have something like da.str.format("sometext: {:2.2f}")

Or maybe da.as_str(...)

headtr1ck · 2023-03-15T09:57:12Z

By the way, this works (but probably slow?):

def tostr(x):
    return f"{:.2f}"

tostr_v = np.vectorize(tostr)


da = xr.DataArray([1, 2, 3.456], dims="x")

tostr_v(da)

husainridwan · 2023-03-15T10:35:06Z

I think the initial issue was to create a method that allows any specific significant figure to be used and then formatted as strings after. I think I'm having issues with a floating dtype. Check the image below

husainridwan · 2023-03-15T11:52:48Z

@k

…stringify

husainridwan · 2023-03-16T06:40:26Z

@TomNicholas & @headtr1ck , Can you review this so I can know what to change? I think it's failing just one test

headtr1ck · 2023-03-16T06:53:52Z

Again, I think that this implementation is too specific and the issue wanted a more general string formatting option.

Also, auch a method is better kept as part of the StrAccessor.

keewis

I also feel that the suggestion here is a bit too specific, so before discussing the implementation (and fixing tests) I think we should figure out a good API. Below I'm suggesting .str.format because that's what the original issue proposed, but really the only requirement I have is that the API feels natural and does not do too much at once.

The original issue also suggested

xr.DataArray("%.3f").str % arr

which would allow having a array of formats, but I feel that's a bit too complicated for now (or in general, not sure).

keewis · 2023-03-16T10:05:59Z

doc/warnings.log

I think this has been added by accident. Could you remove that file?

keewis · 2023-03-16T10:17:52Z

xarray/core/dataarray.py

@@ -6670,6 +6670,68 @@ def resample(
            **indexer_kwargs,
        )

+    def roundStringify(self, n: int) -> DataArray:


we use snake-case for our functions, so it should be round_stringify. However, before you change that I agree with @headtr1ck that the method combines two different things (rounding plus string interpolation).

As an API, I would probably prefer something like this:

( arr.round(0.01) # round to two decimal digits .str.format("%.03f") # display three decimals )

where .str.format can be implemented in a efficient manner using numpy.char.mod.

The advantage of that is that it is a lot more versatile, for example:

ds = xr.Dataset({"a": ("x", ["a", "b", "c"]), "b": ("x", [0.2756, 6.7279, 3.7832])}) ds.a.str.format("%s: ").str + ds.b.str.format("%.03")

@keewis str.format already exists and behaves like the python str.format method (also different from the builtin format() method).

@keewis str.format already exists and behaves like the python str.format method (also different from the builtin format() method).

Good find.
This method is working the other way around as the proposed solution, but it should work perfectly.

Untested:

fmt = xr.DataArray("{:.2f}") da = xr.DataArray([1, 2, 3.456], dims="x") fmt.str.format(da)

the issue with that is that it will "stringify" the values before passing it to str.format, so the only format you can pass it is :s. So I guess the only thing to change is that?

keewis · 2023-03-16T10:30:53Z

xarray/tests/test_dataarray.py

+    def test_roundStringify_output_type(self, my_obj):
+        result = my_obj.roundStringify(3)
+        assert isinstance(result, xr.DataArray), "output should be a DataArray"
+        assert isinstance(result, xr.DataArray), "output should be a DataArray"
+
+    def test_roundStringify_attrs(self, my_obj):
+        attrs = {"units": "meters"}
+        my_obj.attrs = attrs
+        new_da = my_obj.roundStringify(2)
+        assert "units" in new_da.attrs.keys(), "expected attribute not found"
+        assert new_da.attrs["units"] == attrs["units"], "attribute values not matched"
+
+    def test_roundStringify_edge_case(self):
+        # test case when all values are zero
+        zeros = xr.DataArray(np.zeros((3,), dtype=float), dims=["x"])
+        rounded_zeros = zeros.roundStringify(n=3)
+        expected_result = xr.DataArray(["0", "0", "0"], dims=["x"])
+        xr.testing.assert_allclose(rounded_zeros, expected_result)


we usually use a actual-expected pattern (if possible), combined with heavy use of pytest.mark.parametrize.

Edit: oh, sorry, I just saw that the last test already uses that pattern.

For example, the attrs check could become:

@pytest.mark.parametrize("keep_attrs", [True, False]) def test_str_format_attrs(self, keep_attrs): attrs = {"units": "meters"} arr = xr.DataArray(["a", "b", "c"], attrs=attrs) expected = arr.copy() if not keep_attrs: expected.attrs.clear() with xr.set_options(keep_attrs=keep_attrs): actual = arr.str.format("%s") xr.testing.assert_identical(actual, expected)

but of course if we test .str.format it should go into the module that contains the tests for the string accessor (xarray.tests.test_accessor_str, if I'm not mistaken)

Thank you very much. I'll look into the changes and recommendations right away

husainridwan · 2023-03-18T07:10:34Z

@headtr1ck @keewis can you check it now? I’ve modified it a bit

headtr1ck

Since da.str.format already exists, I am not sure how much benefit this function gives.

But apparently it is difficult to discover and use da.str.format, so maybe a simpler interface is useful.

Anyway, I'm the current state the implementation is still too specific.

headtr1ck · 2023-03-18T09:18:06Z

xarray/core/accessor_str.py

@@ -51,6 +51,7 @@

 import numpy as np

+import xarray as xr


No need to import this

headtr1ck · 2023-03-18T09:18:28Z

xarray/core/accessor_str.py

+        array(['1.46e+05', '-3.25e+06', '0.00125'], dtype='<U8')
+        Dimensions without coordinates: x
+        """
+        self._obj = self._obj.astype(float)


Do not change the stored object

This was to ensure all inputs were the same dtype before formatting in case of nan or inf values.

That's ok, but then store it in a temporary, local variable.

headtr1ck · 2023-03-18T09:20:27Z

xarray/core/accessor_str.py

+        Dimensions without coordinates: x
+        """
+        self._obj = self._obj.astype(float)
+        return xr.DataArray(


Use type(self._obj)(...)

Actually, do you even need to wrap it into a DataArray? What does _apply return?

xarray/core/accessor_str.py

husainridwan · 2023-03-20T18:44:13Z

@keewis it seems .str.format does what we want the issue is requesting. Is there any need for the issue or can you clarify exactly what needs to be improved?

husainridwan · 2023-03-30T13:39:12Z

@keewis @TomNicholas @headtr1ck any comments?

…into round-stringify

husainridwan and others added 4 commits March 15, 2023 01:30

Add new method to format DataArray as strings

ffd70ac

Merge branch 'round-stringify' of https://github.com/alrho007/xarray …

89b5e22

…into round-stringify

Add new method to format DataArray as strings

d07a7c4

[pre-commit.ci] auto fixes from pre-commit.com hooks

ec98594

for more information, see https://pre-commit.ci

husainridwan changed the title ~~Add method to format xarray.DataArray as strings after round them to significant figures.~~ Add method to format xarray.DataArray as strings after round them to significant figures for issue #5985 Mar 15, 2023

Merge branch 'main' into round-stringify

655e115

husainridwan closed this Mar 15, 2023

husainridwan reopened this Mar 15, 2023

husainridwan added 5 commits March 15, 2023 13:10

Merge branch 'main' into round-stringify

619bfac

Add changes to xarray.DataArray.roundStringify method

53c68e2

Add changes to xarray.DataArray.roundStringify method

e2db4a2

Merge branch 'main' of https://github.com/alrho007/xarray into round-…

8def641

…stringify

Add changes to xarray.DataArray.roundStringify method

c6d99b6

keewis reviewed Mar 16, 2023

View reviewed changes

Merge branch 'main' into round-stringify

cb51b5e

headtr1ck mentioned this pull request Mar 17, 2023

Formatting data array as strings? #5985

Open

husainridwan added 2 commits March 17, 2023 18:51

Add method to format DataArray as strings for issue pydata#5985

7694dcd

Add method to format DataArray as strings for issue pydata#5985

3ffe339

husainridwan changed the title ~~Add method to format xarray.DataArray as strings after round them to significant figures for issue #5985~~ Add method to format xarray.DataArray as strings for issue #5985 Mar 17, 2023

headtr1ck requested changes Mar 18, 2023

View reviewed changes

Merge branch 'main' into round-stringify

f54e756

husainridwan added 2 commits March 24, 2023 19:35

Merge branch 'main' into round-stringify

36acb4e

Merge branch 'main' into round-stringify

942bcf9

husainridwan added 4 commits April 3, 2023 12:57

Merge branch 'pydata:main' into round-stringify

34d0d96

Modify interp_na to consider decreasing index

095cd30

Merge branch 'round-stringify' of https://github.com/alrho007/xarray …

474d196

…into round-stringify

Modify interp_na for issue pydata#4637

e6e1db3

husainridwan closed this Apr 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add method to format xarray.DataArray as strings for issue #5985 #7628

Add method to format xarray.DataArray as strings for issue #5985 #7628

husainridwan commented Mar 15, 2023 •

edited

Loading

husainridwan commented Mar 15, 2023

headtr1ck commented Mar 15, 2023 •

edited

Loading

headtr1ck commented Mar 15, 2023

husainridwan commented Mar 15, 2023

husainridwan commented Mar 15, 2023

husainridwan commented Mar 16, 2023

headtr1ck commented Mar 16, 2023

keewis left a comment •

edited

Loading

keewis Mar 16, 2023

keewis Mar 16, 2023

DocOtak Mar 16, 2023

headtr1ck Mar 17, 2023

keewis Mar 17, 2023 •

edited

Loading

keewis Mar 16, 2023 •

edited

Loading

husainridwan Mar 16, 2023

husainridwan commented Mar 18, 2023

headtr1ck left a comment

headtr1ck Mar 18, 2023

headtr1ck Mar 18, 2023

husainridwan Mar 18, 2023

headtr1ck Mar 18, 2023

headtr1ck Mar 18, 2023

headtr1ck Mar 18, 2023 •

edited

Loading

husainridwan commented Mar 20, 2023

husainridwan commented Mar 30, 2023

Add method to format xarray.DataArray as strings for issue #5985 #7628

Add method to format xarray.DataArray as strings for issue #5985 #7628

Conversation

husainridwan commented Mar 15, 2023 • edited Loading

husainridwan commented Mar 15, 2023

headtr1ck commented Mar 15, 2023 • edited Loading

headtr1ck commented Mar 15, 2023

husainridwan commented Mar 15, 2023

husainridwan commented Mar 15, 2023

husainridwan commented Mar 16, 2023

headtr1ck commented Mar 16, 2023

keewis left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

keewis Mar 17, 2023 • edited Loading

Choose a reason for hiding this comment

keewis Mar 16, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

husainridwan commented Mar 18, 2023

headtr1ck left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

headtr1ck Mar 18, 2023 • edited Loading

Choose a reason for hiding this comment

husainridwan commented Mar 20, 2023

husainridwan commented Mar 30, 2023

husainridwan commented Mar 15, 2023 •

edited

Loading

headtr1ck commented Mar 15, 2023 •

edited

Loading

keewis left a comment •

edited

Loading

keewis Mar 17, 2023 •

edited

Loading

keewis Mar 16, 2023 •

edited

Loading

headtr1ck Mar 18, 2023 •

edited

Loading