Replace pyke package with other solutions for netcdf loading

[pp-mo]

I think Pyke is near-obsolete, at best static with no development since 2010 as fas as I can tell
This is a bit worrying, to say the least.

We could replace it with another Python logic programming solution, but they all seem in a similar state IMHO (check it out).

Instead, I reckon that the actual rules operation is really quite simple and not hard to rewrite in ordinary Python. Which would certainly make the rules easier to maintain.

BTW, I don't think we are really making ideal usage of the logic programming concept in the loader anyway : A rules-type language would be a really good solution to assigning netcdf variables to specific CF categories. This is currently handled by the iris.fileformats.cf wher the CfReader using the identify methods in the various subclasses of CFVariable.
Either that is a missed opportunity or, possibly, it just shows that pure Python solutions are actually perfectly adequate !

[pp-mo]

Detail : state of play on possible logic programming packages.

Pyke

• Still on sourceforge, now pretty ancient in itself !
• latest, and only public, release in 2010
• 7 comments in last 2 years

Best of others seem to be logpy(or 'kanren' ?) and pyswip
Both on github. Both have better syntax. But neither is seeing much action...

• logpy releases 0.2.1 and 0.2.2, both on 1 Dec 2015
• pyswip releases
  • 0.2.5/6/7 May-Jun 2018
  • 0.2.8 Feb 20 2019 : doc-changes only, nothing functional

[bjlittle]

@pp-mo Guilty as charged.

I've been meaning to purge pyke for a long time now. In fact, I've even got a branch with it half done... just not had the time or capacity to finish the job 😢

I ended up taking the pure Python route, but still building on the iris.fileformats.cf layer.

It would be very healthy of us to purge ourselves of the pyke dependency, as it is effectively a dead package, and I'd be able to sleep better at night. However, it would be great to get ASV in place beforehand, so that we can measure the performance impact...

Nudge @trexfeathers 😉

[bjlittle]

That said, I'd even float the concept of making iris.fileformats.cf a standalone package, e.g., akin to cf-units.

Indeed I'd even go as far as saying that we should seriously consider how we trim the fat of iris to slim it down to a leaner core, with opt in pluggable functionality e.g., not everyone wants to plot, use PP, or FF, or GRIB or indeed NetCDF... which reminds me that we shouldn't be shipping the mahoosive test suite within the iris package. The conda-forge recipe rips it out for good reason.

[bjlittle]

Couldn't help myself, see #3416

[pp-mo]

However, it would be great to get ASV in place beforehand, so that we can measure the performance impact...

At the very least, we do need to be very sure of any performance impact.

[pp-mo]

making iris.fileformats.cf a standalone package, e.g., akin to cf-units.

That's surely a different issue though ??

On the subject, I'm not convinced that a standalone 'cf' will be useful without the rules too.
We can probably make iris.fileformats.netcdf standalone.
However, the "easy" way will be like iris-grib, not like cf-units.
The difference being that iris-grib is unashamedly iris-specific (it builds cubes), whereas cf-units is a tidy independent chunk of functionality.
To make a iris-independent file-format loader will mean re-writing it to target an intermediate result format which is independent of Iris classes. It can be done, but it's less easy and even messy -- e.g. it must still allow construction of 'DataProxy's for lazy data access.

[bjlittle]

PyKE takes a HyKE... go for it @pp-mo 👍

[rcomer]

Is this closed by #4198?

[pp-mo]

Is this closed by #4198?

Yes.
Thanks !