Warning messages about `getfattr ... | grep -c '.daos'`

[ScottWales]

Describe the bug

I'm seeing extra warning messages in CSET logs that look like this:

sh: -c: line 0: syntax error near unexpected token `)'
sh: -c: line 0: `getfattr /home/562/saw562/cylc-run/cset-metplus-test/run12/share/cycle/20240203T0000Z/data/1/)scratch)dx2)gay548)ML)AIFS)outputs)postproc)ensv1)AIFS)ensv1)20240203T000000)ifs)0p25)fc)z_20240203T000000_6h_360_em1.nc | grep -c '.daos''

The error appears to be from the netcdf library passing the path unescaped through a shell function at https://github.com/Unidata/netcdf-c/blob/8b11cd27bd3b95fa7b2adecd174ddd6fbca3f901/libdispatch/dinfermodel.c#L1648

Perhaps it would be better to use urllib,parse.quote_plus() to create an escaped path when linking in the data in the fetch_fcst tasks

How to reproduce

Steps to reproduce the behaviour:

1. Run the cset workflow at head of trunk
2. Check the log files in ~/cylc-run/cset-metplus-test/runN/log/job/20240203T0000Z/bake_recipes/NN/bunch.0.err

Expected behaviour

There shouldn't be any 'sh' errors

[jfrost-mo]

Good job finding that. I'd seen that error, but couldn't trace down where it was coming from.

Fortunately, despite erroring here it falls back to the correct path for our use case, so other than the extra log messages it continues working. I'll send a patch upstream to netcdf-c, and also adopt the different quoting you suggest.

[jfrost-mo]

Just to check, what version of libnetcdf-4 do you have installed? Supposedly this was fixed in the 4.10 release. Unidata/netcdf-c#3196

[ScottWales]

Looks like we're using 4.9.3, I've got metplus in the same conda environment which has a version restriction on libnetcdf, trying to update netcdf gives:

Error:   × failed to solve requirements of environment 'default' for platform 'linux-64'
  ├─▶   × failed to solve the environment
  │
  ╰─▶ Cannot solve the request because of: metplus >=6.2 cannot be installed because there are no viable options:
      ├─ metplus 6.2.1 | 6.2.1 | 6.2.1 | 6.2.1 | 6.2.1 | 6.2.1 | 6.2.1 | 6.2.1 would require
      │  └─ metview ==5.26.2, which cannot be installed because there are no viable options:
      │     └─ metview 5.26.2 | 5.26.2 would require
      │        └─ libnetcdf >=4.9.3,<4.9.4.0a0, for which no candidates were found.

[jfrost-mo]

Having a look at it, I'm not liking the URL quoting that much, as it makes it hard to read the full path, which is the intent behind these filenames over a UUID or similar.

>>> s.replace("/", ")")
')data)scratch)user.name)test-data)20000101T0000Z)example.nc'
>>> urllib.parse.quote_plus(s)
'%2Fdata%2Fscratch%2Fuser.name%2Ftest-data%2F20000101T0000Z%2Fexample.nc'

As a compromise, how about changing the delimiter character to something that isn't a shell metacharacter, such as }?

>>> s.replace("/", "}")
'}data}scratch}user.name}test-data}20000101T0000Z}example.nc'

This should prevent these error messages appearing (and other unquoted problems) while not being any harder to read.