compass python package (compass v1.0) #28

So far, only the QU240 mesh is supported. Also, add `scipy` as a dependency.

This will allow us to initializes later with other initial conditions.

For these changes, the logging module has been moved to mpas_tools so the same context manager can be used there, along with the check_call command that suports logging.

The fix is to rename one of the repeats. Otherwise, xarray is pretty unhappy.

The reorganization: * adds support for QU240wISC (and other meshes with ice-shelf cavities) * separates mesh and init test cases * adds support for 4 initial conditions: * PHC * PHC with BGC * EN4_1900 * EN4_1900 with BGC * Adds a subdirectory for the initial condition. Each testcase other than mesh is within this subdirectory * Added the full matrix of QU240 test cases (maybe overkill?): * with and without ice-shelf cavities * PHC and EN4_1900 * with and without BGC * split_explicit and RK4

The new test suite has the same tests as the QU240 in nightly suite, but for the QU240wISC mesh instead.

The key ``steps_to_run`` has been added to the ``testcase`` dict and individual steps run the same way as the whole testcase except that there is only one step in ``steps_to_run`` and logging is handled slightly differently. This means that config options related to the requested number of cores, minimum cores, threads, etc. can be parsed at the testcase level, then the number of cores to run with can be determined in ``run_steps`` before running individual steps. Testcases that run all steps in the order they are added can just leave ``steps_to_run`` with its default value. If there are some optional steps that should be excluded, the testcase will need to replace the default list with an appropriate list of steps.

This also includes adding config options related to mesh provenance that will eventually be included in all NetCDF output.

Both the suite name and the tests within it need to be updated.

Also, add support for namelist and streams replacement dicts.

The files for diagnostics are missing because some work is needed in MPAS-Tools to support this step. Also add a suite for testing QU240 and QUwISC240 spin-up and files for E3SM.

This merge also adds git as a dependency.

Individual steps were being logged to their own log files but are now written to the same log file within case_output for the whole test case.

They have different depth conversions and could, in theory, have different variable names, dimensions, etc.

It has some common config options that users might want to change.

Fix some other output and logging issues.

This requires setting the target and min cores in setup() for several types of steps in global_ocean that were only setting these values in the test case's run()

This overrides the config option `mpas_model` in `[paths]` for convenience of setting the MPAS-Model directory

This improves the formatting of the help display.

Add a comment that new cores need to be added.

provenance.write was missing a docstring. cull_mesh had a mis-formatted list. config.get_source_file had an incorrectly formatted :ref: tag

Also, improve how the compass package is found (with "include", not "exclude").

Build the docs after the compass package, since this will be needed for the autogenerated API.

Since we almost always want to: * update PIO * make a graph partiton * run the model these first two are now an optional part of running the model. By having run_model take `step`, we can get information like the number of cores, threads and work dir. from there. To make this easier, the core and configration were added to `step` while setting up `testcase` defaults. There could be a little confusion between "core" and "cores" (and "configuration" and "config file" for that matter) but this is hard to avoid.

The option has been added to pass a list of tuples of replacement files and their modules so some of the tedium of updating namelists and streams can be handled internally.

This includes modifications to how databases and downloads are handled, requiring updates so many config files.

The API is now quite different. The `collect()` functions for each test case and step now require fixed arguments. There is no need to call `setup_testcase_default` and `setup_step_default` so these have been hidden. Instead, the framework calls `collect()`. Configurations should add test cases with `add_testcase` and test cases should add steps with `add_step`. If a test case changes its subdirectory in `collect()`, it should call `set_testcase_subdir` to update both `subdir` and `path`.

Add a test suite with all examples

There were problems with a nonexistant mixed-layer depth output stream. The stream now exists but doesn't write anything.

This requires a new version of geometric_features (0.3.0).

Give the number of tests that failed.

Rearrange the directory structure so there are fewer subdirectories for split_explict tests and better organization for RK4 tests.

The 42layerWOCE and 60layerPHC interfaces were incorrect -- they were layer midpoints. The 60layerPHC has been corrected using the binary data before rounding to compute interface locations such that their averages reproduce the midpoints. I was not able to find the bindary 42layerWOCE data. As this veritcal grid is not used, I am simply removing it.

Remove tests from SIA stuite that aren't in the legacy version of the suite.

This affects nearly all config files at the configuration level

Move example_ocean.cfg to example_configs/ocean.cfg

Add missing pyamg Add more to setup.py Add pip check

Three namelist options have new names to accommodate partial top cells. The new names are: config_init_vertical_grid_type config_alter_ICs_for_pcs config_pc_alteration_type

Without this, users don't have a way of telling compass to use less than the whole machine.

Make most functions from the testcase module into methods of ``TestCase``.

Move the template for step run scripts to the `step` package.

This makes them easier to refer to in docstrings

Code for adding and generating namelists and streams files has been moved into the Step class, since is relies heavily on the step's attributes. The function for processing input files has also been made non-public and is called as part of generate(), not externally in the setup module.

This is needed so steps know their MPAS core at construction and can use that information to name namelist and streams files.

The name of the test case isn't always unique so it is necessary to use a unique key like subdir (or path) With this merge, only the values of the test_cases and steps dicts are used in the framework but the dict datastructure ensures that each test case has a unique subdirectory Steps remain indexed by name becasue this is useful for finding a specific step in a test case by name. This requires that each step have a unique name (for now, the same as the subdir).

This makes it easier to link to files in other test cases without having to know the relative path between them.

Spinup has been renamed to dynamic_adjustment and suites need to be updated accordingly.

It hasn't proven to be very useful, compared with "real" examples for MPAS cores.

The code is a little more verbose but more intuitive this way.

The docstring for Step.generate and TestCase.generate was not very clear, so they have been made more verbose.

They incorrectly referred to the old step dictionaries

Add a `legacy_` prefix to all so they don't clash with new documentation as it is added.

Also update some authors and the year for the copyright.

Otherwise, it will be hard ot add the same structure later for the developer's guide.

Update ice_shelf_2d and global_ocean accordingly

Add instructions on building the package and then the docs. Update the sphinx config to use the compass version from the package. Add the developer's guide to the main TOC.

Add an overview to developer's guide; add some hooks from the user's guide

following recent framework changes

following recent framework changes.

Correct documentation of 60layerPHC grid

This merge includes a large number of updates to the User's Guide following recent moves to classes and a new naming convention. Machine usage has also been updated to be consistent with recent testing with E3SM modules. The Developer's Guide has also been updated with the change from "configuration" to "test group".

Add a glossary.

Commits on Apr 16, 2021

Fix MPAS-Ocean --> MALI in landice.cfg

xylar committed Apr 16, 2021

Configuration menu

View commit details

Copy full SHA for 14fabda

Browse repository at this point

Copy the full SHA

14fabda View commit details

Browse the repository at this point in the history

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

compass python package (compass v1.0) #28

compass python package (compass v1.0) #28

Commits on Apr 14, 2021

Commits on Apr 15, 2021

Commits on Apr 16, 2021

Commits on Apr 18, 2021

Commits on Apr 23, 2021