Add capability to add new kind definitions to capgen #713

nusbaume · 2026-01-16T19:40:22Z

Adds the ability for the host model to provide additional kind definitions to the auto-generated ccpp_kinds.F90 file.

This PR provides the ability for a host model, during the creation of the cap in capgen, to provide a dictionary of kind types (the kind type name, module, and specification/name within the module), which then add the kind to ccpp_kinds.F90, and thus make it accessible throughout the CCPP cap. An example of this dictionary being used in practice can be found in a branch of CAM-SIMA here:

https://github.com/nusbaume/CAM-SIMA/blob/pumas_round3/cime_config/cam_autogen.py#L497-L504

Additional info about what this PR is specifically trying to fix can also be found in issue #712

User interface changes?: Yes

The kind_types argument for CCPPFrameworkEnv has been changed from a string to a dictionary. If one is using command-line arguments then it the kind_types argument has changed from kind_type=kind_spec to kind_type=kind_module,kind_spec, with a colon-delimiter for multiple definitions. Examples should be findable in the comments/testing code provided in this PR.

Fixes:

Fixes #712 - Need ability to expose scheme-defined kind to cap in capgen

Testing:
test removed: N/A
unit tests: Updated python tests to properly use the new kind_types dictionary structure.
system tests: Added new kind to capgen_test to ensure this fucntionality works as expected.
manual testing: Tested this functionality in CAM-SIMA with the PUMAS cloud microphysics scheme (which has its own kind definition).

climbfuji

Looks reasonable to me

climbfuji · 2026-01-20T19:41:51Z

scripts/framework_env.py

+            result[key.strip()] = value.split(',')
+        return result
+
+    parser.add_argument("--kind-types", type=parse_kind_dict,


I think that a more modern way of specifying multiple kind types on the command line would be to allow multiple occurrences of --kind-type. But there may be other argument for capgen that use semicolons to pass multiple values to an argument.

--kind-type kind_r8=iso_fortran_env,real64 --kind-type kind_phys=iso_fortran_env,real32 ...

gold2718

@nusbaume, thanks for addressing this need!

I have several small concerns which I document inline. I also have a few larger concerns:

Although not explicitly documented, the --kind-type argument to ccpp_capgen was developed as a way to make it easy for a host model to redefine basic CCPP kinds. The case documented in #712 seems to me to be the opposite of that in that this type was created explicitly to avoid overriding by the host model (or at least to insulate PUMAS internal data from changes to the r8 kind).
This PR forces host model builders to know and document any kinds defined in the suites they happen to be using for any experiment. I suppose they could include kinds that are not in use but that could be confusing and error prone.
The CCPP style is to document data and interfaces in the metadata. I feel that these new types should be somewhere in the suite's metadata -- ideally in the metadata associated with the module that defines the new kind. If the group sentiment agrees with this philosophy, I have a branch I could submit as a PR to this PR branch that adds a metadata table to the kinds module.
I like the old --kind-type user interface and would like to return to it. It did not need any module information since it was using well-known kinds. Still, if we want the ability to use kinds that are not standard (why?), I can see modifying the RHS while still retaining the multiple use syntax (as also suggested by @climbfuji).

gold2718 · 2026-01-18T20:54:34Z

scripts/ccpp_capgen.py

+        msg = f'Writing {KINDS_FILENAME} to {output_dir}'
+        run_env.logger.info(msg)


Is lazy-style formatting no longer preferred for logging messages (to avoid unnecessary formatting)?

gold2718 · 2026-01-18T21:00:14Z

scripts/framework_env.py

+        if kind_type in self.__kind_dict:
+            # The kind module should always be
+            # the first element in the list:
+            kind_mod = self.__kind_dict[kind_type][0]


Do we need a check here to make sure we have an iterable with at least one item or some other sanity check? I suppose a failure here would be an internal error but we might get a cleaner one here by checking.

gold2718 · 2026-01-18T21:01:21Z

scripts/framework_env.py

-            kind_spec = self.__kind_dict[kind_type]
+            # The kind specification should always be
+            # the second element in the list:
+            kind_spec = self.__kind_dict[kind_type][1]


Do we need a check here as well?

gold2718 · 2026-01-18T21:05:43Z

scripts/framework_env.py

+
+    parser.add_argument("--kind-types", type=parse_kind_dict,
+                        metavar="kind_types", default=dict(),
+                        help="""Data size for real(<kind_types>) data.


Why just real types? For instance, doesn't PUMAS define a kind for integers?

gold2718 · 2026-01-18T21:07:52Z

scripts/framework_env.py

+        dict_entries = arg_str.split(':')
+        for entry in dict_entries:
+            key, value = entry.split('=')
+            result[key.strip()] = value.split(',')


Should value be checked to make sure it has exactly one comma?

gold2718 · 2026-01-18T21:19:28Z

scripts/framework_env.py

-<kind_val> SHOULD be a valid ISO_FORTRAN_ENV type""")
+    # Define parser for kind dictionary input
+    def parse_kind_dict(arg_str):
+        """Parse a 'key1=value1:key2=value2' string


Would doctests be useful here? If I am reading the code correctly, this function is used by the argparse library which makes it responsible for input-data sanitation for this keyword. Am I thinking about it the wrong way?

nusbaume · 2026-01-21T23:38:43Z

@gold2718 Ahh, my mistake! I just assumed that the --kind-type was for anything kind-related, but if it is specific to changing the general CCPP kind options then we in CGD are fine to go with the metadata route instead (it should certainly work for the PUMAS use case). Also if we do go that route then I am happy to revert all of my kind_types changes, which I think would resolve most of the other comments/requests here?

gold2718 · 2026-01-22T09:07:04Z

@gold2718 Ahh, my mistake! I just assumed that the --kind-type was for anything kind-related, but if it is specific to changing the general CCPP kind options then we in CGD are fine to go with the metadata route instead (it should certainly work for the PUMAS use case).

I think mistake is too strong a word here -- it's not like there is some documentation you forgot to read :)
At the time, I thought the scope was to be able to modify internal CCPP framework kinds (e.g., kind_phys) and possibly to introduce separate host-model kinds (so the host cap could emit the correct code). The biggest flag I saw with your change is that it allows the user to easily change a kind which the scheme explicitly does not want changed.

Also if we do go that route then I am happy to revert all of my kind_types changes, which I think would resolve most of the other comments/requests here?

I can add this to my PR as I already have the mods necessary to add the kind to the metadata (with some tests). The biggest issue I have with my mods is that they allow a module metadata header into the suite world. Once it is visible, we need to think about whether or not there might be unintended side effects.

dustinswales

@nusbaume Thanks for adding this feature.
I don't have a strong opinion on kind-types vs. kind-type. We use kind-type in the CCPP SCM, but switching to kind-types would be trivial.

peverwhee · 2026-01-28T19:17:42Z

@gold2718 i'm not sure what you mean by:

The biggest issue I have with my mods is that they allow a module metadata header into the suite world. Once it is visible, we need to think about whether or not there might be unintended side effects.

But I'm happy to comment once you open the PR (at which point hopefully I understand the context!)

nusbaume added 3 commits January 14, 2026 16:06

First attempt at extending CCPP kinds file.

d7122ea

Add kind addition test to 'capgen_test' collection.

f1c2b73

Update comments in 'temp_kinds.F90'

4ae3fa9

nusbaume self-assigned this Jan 16, 2026

nusbaume requested review from a team and gold2718 as code owners January 16, 2026 19:40

nusbaume added enhancement capgen bugs, requests, etc. that involve ccpp_capgen labels Jan 16, 2026

nusbaume added 3 commits January 16, 2026 12:42

Merge remote-tracking branch 'upstream/develop' into pumas_kind_add

098cb1b

Fix failing variable transform tests.

15bae29

Fix failing doctests.

e18d907

climbfuji approved these changes Jan 20, 2026

View reviewed changes

gold2718 requested changes Jan 20, 2026

View reviewed changes

dustinswales approved these changes Jan 22, 2026

View reviewed changes

		msg = f'Writing {KINDS_FILENAME} to {output_dir}'
		run_env.logger.info(msg)

Add capability to add new kind definitions to capgen #713

Are you sure you want to change the base?

Add capability to add new kind definitions to capgen #713

Uh oh!

Conversation

nusbaume commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

climbfuji left a comment

Choose a reason for hiding this comment

Uh oh!

climbfuji Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gold2718 left a comment

Choose a reason for hiding this comment

Uh oh!

gold2718 Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

gold2718 Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

gold2718 Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

gold2718 Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

gold2718 Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

gold2718 Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

nusbaume commented Jan 21, 2026

Uh oh!

gold2718 commented Jan 22, 2026

Uh oh!

dustinswales left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peverwhee commented Jan 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

nusbaume commented Jan 16, 2026 •

edited

Loading

climbfuji Jan 20, 2026 •

edited

Loading

dustinswales left a comment •

edited

Loading