Bugs Related to Serialization

[andrewellis55]

Summary

My team and I have been developing a web application (based on plotly dash) that uses openmdao quite heavily in the backend. In order to pass objects between callbacks we need to serialize them and then de-serialize them. We're currently looking to upgrade from openmdao 3.17 to 3.30 and these are two fixes to issues we encountered related to serialized openmdao objects and passing them between processes. I know this probably isn't too common of a use case, but I'd appreciate if these changes could be accepted.

1. weakref serialization

Serialization of a weakref breaks the reference. After a weakref is broken, it becomes a function that returns None. This is shown in the following code

import weakref
import gc

class System:
    pass

system = System()
_system = weakref.ref(system)

# System is deleted and garbage collectes
del system
gc.collect()

# After this deletion and garbage collection _system() will return None
assert _system() == None

This case is not accounted for in the System._setup solvers function logic. The RuntimeError error will always be raised as self._system() is None, not self._system.

        if self._system is None:
            self._system = weakref.ref(system)
        elif self._system != weakref.ref(system):
            raise RuntimeError(f"{type(self).__name__} has already been assigned to "
                               f"{self._system().msginfo} and cannot also be assigned to "
                               f"{system.msginfo}.")

This was corrected as follows

        # Default initialization
        if self._system is None:
            self._system = weakref.ref(system)
        # Following Dead Weakref
        elif self._system() is None:
            self._system = weakref.ref(system)
        # Assignment Mismatch
        elif self._system != weakref.ref(system):
            raise RuntimeError(f"{type(self).__name__} has already been assigned to "
                               f"{self._system().msginfo} and cannot also be assigned to "
                               f"{system.msginfo}.")

2. self.matrix_free is _UNDEFINED

The is function can be a little finicky when working with multiprocessing and passing an object between processes. We found instances where the object would be created in one process, then passed to another and self.matrix_free is _UNDEFINED would return False but self.matrix_free == _UNDEFINED would return True. This is because the _UNDEFINED class it's referencing is a class from another process (thus making is False) but they are equivalent (thus making == True). This therefor broke the configure stack for us.

I've made the change in ExplicitComponent and ImplicitComponent _configure to fix this.

Related Issues

• Resolves #

Backwards incompatibilities

None

New Dependencies

None

[coveralls]

coverage: 89.764% (-0.3%) from 90.045%
when pulling fa39187 on andrewellis55:weakref-bug-with-serialization
into 6b42683 on OpenMDAO:master.

[swryan]

would you be able to add a unit test?

[swryan]

thanks!