Training Donut raises "Tensor had NaN values"

[WGierke]

While training Donut on Synthetic Shift Outliers an exception occured:gradient for model/donut/p_x_given_z/mean/dense/bias:0 has numeric issue : Tensor had NaN values [[Node: quiet_donut_trainer_9/CheckNumerics_13 = CheckNumerics[T=DT_FLOAT, message="gradient for model/donut/p_x_given_z/mean/dense/bias:0 has numeric issue", _device="/job:localhost/replica:0/task:0/device:CPU:0"](quiet_donut_trainer_9/clip_by_norm_13/truediv)]]

[maxifischer]

It looks like the gradients become 0, something is going wrong in the training. Setting optimizer_params = {epsilon:1e-05} in QuietDonutTrainer (increasing epsilon for AdamOptimizer) removed the error for me. But it's worth a discussion whether and how much we should improve the parameters of Donut.

[WGierke]

Solved via #82