[WIP] Fidelity op

[MichaelBroughton]

@jaeyoo would you be able to double check what's going wrong with the adjoint implementation of the gradient for fidelity ? It seems like this formula should be correct, but the autograd functionality for TF isn't lining up with the manual adjoint calculations. This isn't super urgent for the 0.5.0 release, but it seems pretty close so might be worth trying to see if we can merge it before release.

[jaeyoo]

@MichaelBroughton Yes right.. in fact, this is what I am stuck with these days. So my current workaround is just add custom_gradient like : tf.math.conj(dip) * ip + tf.math.conj(ip) * dip where dip = tape.gradient(ip, symbol_values) and it works... In details,

def grad():
  w = tf.identity(symbol_values)
  with tf.GradientTape() as tape:
    tape.watch(w)
    ip = inner_product_op.inner_product(
        circuits_tensor, symbol_names, w, other_circuit_tensor)
  dip = tape.gradient(ip, w)
  dfidelity = tf.math.real(tf.math.conj(dip) * ip + tf.math.conj(ip) * dip)
  return dfidelity

[MichaelBroughton]

Ahhhh that's not good. What worries me is that fidelity is just simple transformation around inner_product (tf.math.abs followed by **2) I am a little worried this means the gradient functionality for inner_product may not implement the chain rule correctly. Do you think that is possible ? Do you have a custom_gradient version that also accounts for the downstream gradients ?

[jaeyoo]

Ahhhh that's not good. What worries me is that fidelity is just simple transformation around inner_product (tf.math.abs followed by **2) I am a little worried this means the gradient functionality for inner_product may not implement the chain rule correctly. Do you think that is possible ? Do you have a custom_gradient version that also accounts for the downstream gradients ?

Ahhh yeah in that case, I tested again and realized that the real fix for the issue is what you did: tf.zero_like( ..., dtype=tf.complex64). It's what I couldn't understand the cause from the weird error message. now I can use abs()**2 in my gradient, and I see the test well passed in my newly opened PR. #554. Please look around it.