log_pdf in StocahsticGraph's get_output has the same size with the output tensor(Branch model)

[ssydasheng]

In branch model,
the log_pdf in StocahsticGraph's get_output has the same size with the targeted output tensor, which differs from the master setting, in which the log_pdf will be shape (batch_size, n_samples) by reduce_sum.

In #42 of ../zhusuan/variational.py

    lower_bound = model.log_prob(latent_outputs, observed, given) - \
        sum(latent_logpdfs)_

will crash because of adding latent_logpdfs of different shapes.

[ssydasheng]

@thjashin

[thjashin]

Hi @ssydasheng ,
Actually that's what I intended to. In the new api, there is no further assumptions about shape of the Tensors. no (batch_size, n_samples) anymore. With new StochasticTensors, you can sample in any sample_dim you want. You can take a look at examples/vae.py. Over the days I have thought about for a graphical model, there is nothing special to batch_size, it's just a plate (a rectangle). So my choice in vae.py is just build the model (p(x,z)) like the graphical view (z_mean as (n_z,), z_logstd as (n_z,), z as (batch_size, n_z)), what I add to make it more suitable in variational setting is just adding a n_particles (i.e. n_samples) dimension before all of these shapes, which can be seen as we allow multiple samples (particles) to pass through a node in the graphical model. That meets the need when getting q(z) samples of shape (n_particles, batch_size, n_z) so that these samples can be passed into models.

Normal and Bernoulli actually means independent Normal and independent Bernoulli. Discrete will sum over the last dimension. If you look at my new vae under examples/, it sums logpdf by hand over the dimension you want and then gets passed to advi. I will further change the api of advi to make this more comfortable.

For your case, if w is of shape (n1, n2), I guess you can define w_mean, w_logstd, w as shape (n_particles, n1, n2), qw_mean, qw_logstd can be as shape (n1, n2), and qw=Normal(qw_mean, qw_logstd, sample_dim=0, n_samples=n_particles)

Above may be a little messy to understand, we can chat in details.