patching of fortran types only works for first occurence

[rainbowsend]

Given the namelist file setup.nml

&setup
 test%var1 = 1
 test%var2 = 2
/

and the patch

patch={'setup': {'test':{'var2': 3}}}

the call

f90nml.patch('setup.nml', patch, 'patched_setup.nml')

does not patch anything.

However, patching the first occurrence of test% in setup.nml works

patch={'setup': {'test':{'var1': 3}}}

[marshallward]

Sorry for taking so long to reply, this came in at a busy time for me.

I can replicate this. Although patch() currently has a lot of limitations, I would have expected this one to work. My guess is that traversal of the derived types is very poor.

I will try to find some time to look into this.

[ccarouge]

We are using f90nml with derived-types as well and we have a related but different behaviour. With the example above, if we read in the namelist first and then apply the patch, the patch replaces the whole entry for the derived-type:

mynml = f90nml.read('setup.nml')
mynml.patch(patch)
print(mynml)
&setup
    test%var2 = 3
/

test%var1 disappears.

I'm guessing you want the same behaviour from patch() from both call patterns and neither of these behaviours works as expected.

[marshallward]

Hi @ccarouge!

I agree this is mostly like the same issue. I (finally) had a look here and it seems that the aggressive .pop() of values off of patch_nml were causing variables to get removed too early.

The scanner has become such a mess that I can't even pretend to understand what is going on, but this patch does appear to fix the issue:

diff --git a/f90nml/parser.py b/f90nml/parser.py
index a1d3eca..ac91048 100644
--- a/f90nml/parser.py
+++ b/f90nml/parser.py
@@ -526,7 +526,7 @@ class Parser(object):
             # Check for value in patch
             v_patch_nml = None
             if v_name in patch_nml:
-                v_patch_nml = patch_nml.pop(v_name.lower())
+                v_patch_nml = patch_nml.get(v_name.lower())

             if parent:
                 vpar = parent.get(v_name.lower())

I can sort of guess what is happening here: derived type contents should not be pop-ed since they need to be compared over all elements, but it can't be that simple.

At worst, I can just apply this patch and hope that it fixes the problem.

[marshallward]

I've pushed this fix. Hopefully it is actually a fix. I'll do an updated release soon.

[ccarouge]

Thanks @marshallward. For the moment, we found a workaround in pydantic:

# ======================================================
# Copyright (c) 2017 - 2022 Samuel Colvin and other contributors
# from https://github.com/pydantic/pydantic/blob/fd2991fe6a73819b48c906e3c3274e8e47d0f761/pydantic/utils.py#L200

KeyType = TypeVar('KeyType')


def deep_update(mapping: Dict[KeyType, Any], *updating_mappings: Dict[KeyType, Any]) -> Dict[KeyType, Any]:
    updated_mapping = mapping.copy()
    for updating_mapping in updating_mappings:
        for k, v in updating_mapping.items():
            if k in updated_mapping and isinstance(updated_mapping[k], dict) and isinstance(v, dict):
                updated_mapping[k] = deep_update(updated_mapping[k], v)
            else:
                updated_mapping[k] = v
    return updated_mapping

# ======================================================

This seems to work for our use. But happy to use the update from f90nml once it is fixed.