Enh: Adding model package download functions to loads

[NickGeneva]

Earth-2 MIP Pull Request

Description

Fixes: #54

This PR does a few items, however the primary objective is to enable auto-downloading of model packages that are known but are not present on the disk. This PR:

• Adds package download functions into each of the three known models, must be triggered with the prefix e2mip:// on the model name
• Add loader functions for just Pangu_6 and Pangu_24hr
• Adds support for using earth2mip entry points to discover load functions. This means people can add their own model via a python package that injects itself into earth2mips entry point group earth2mip.networks. This eliminates the requirement for metadata.json in favor of pure python. This is how the known models are discovered.

Now running a ensemble config without a any of these models in the model registry just works.

Notebooks could be updated but they do provide good info about how these packages are set up. Should probably get rid of the sub-processes to wget commands to pythonic process. But one step at a time.

Open questions: How should we test this?

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.
• The CHANGELOG.md is up to date with these changes.
• An issue is linked to this pull request.

Dependencies

[NickGeneva]

/blossom-ci

[nbren12]

Thanks. Users will indeed find this very useful, but I am not comfortable with the amount of I/O specific concerns in the packages, or with the backwards incompatible changes to Package.__init__.

Package should abstract such concerns and hide them from the load functions. I think a possible usage could be this:

package = Package("ngc://nvidia/modulus/modulus_dlwp_cubesphere/v0.1:zip")
earth2mip.networks.dlwp(package)

I'll have to think more about the entrypoint mechanism for defining model metadata. For CLI use, we already have a similar mechanism here:

earth2mip/earth2mip/_cli_utils.py

Line 21 in 5999d7e

def add_model_args(parser: argparse.ArgumentParser, required=False):

. This is used by the lagged ensembles script. For API use, I'm not sure the auto-discovery is needed.

[NickGeneva]

/blossom-ci

[NickGeneva]

/blossom-ci

[NickGeneva]

/blossom-ci

[NickGeneva]

/blossom-ci

[NickGeneva]

/blossom-ci

[NickGeneva]

/blossom-ci

[nbren12]

Looks good. Just a couple minor comments, but I am hitting approve. Feel free to merge after addressing them. I don't think @yairchn needs to review this.

[NickGeneva]

/blossom-ci