Not able to pickle dump tadgan pipeline

[dyuliu]

• Orion version: v0.1.5
• Python version: 3.6
• Operating System: macOS 11.2.1

(1)
I tried to use pickle dump tadgan pipeline instance created here
https://github.com/signals-dev/Orion/blob/23f2bccb057572cb244631ca79ba3b623c6080f0/orion/analysis.py#L30

Then an error message from TensorFlow (v1.14) came out:
NotImplementedError: numpy() is only available when eager execution is enabled.

(2)
For other non-tadgan pipeline, so far so good.

[sarahmish]

Based on the keras adapter in MLPrimitives, we are in need to specify the __reduce__ attribute for the object. However, this is a low level operation and setting __getstate__ and __setstate__ is the recommended approach.

Two hurdles to consider:
(1) how to save the optimizer TadGAN.optimizer.
(2) the interpolated layer RandomeWeightedAverage is not recognized and causes ValueError: Unknown layer: RandomWeightedAverage error when loading the pipeline once again.
(3) similarly the custom loss functions are not recognized.

To solve this issue, we can specify custom_object in keras.load_model however, the architecture currently written overrides gradient_penalty loss for critic_x and critic_z.

Current solution is to use the TadGAN pipeline for predictions alone. In the new version of TadGAN #161, I will ensure that continuing training the pipeline is a supported feature.