Optimizing ELBO with respect to variational model only

[AlphaFrank]

Hello,

I am currently doing a course project on optimizing ELBO with respect to variational (inference) model only.

For this following example,
https://github.com/blei-lab/edward/blob/master/examples/sigmoid_belief_network.py
I fixed the model parameters by adding trainable=Fasle in line 76 and 80. Then I printed out the negative ELBO, and found it increasing steadily for each iteration. However, when I fixed the variational parameters, the negative ELBO decreases as it is supposed to.

Is this normal since the ELBO and gradients are estimated? Any help will be appreciated!