vectorising likelihood computations

[yebai]

There are certain cases where the efficiency of HMC sampler can be substantially improved, such as the following example

@model nbmodel begin
  theta ~ Dirichlet(alpha)
  phi = Array{Any}(K)
  for k = 1:K
    phi[k] ~ Dirichlet(β)
  end
  for m = 1:M
    z[m] ~ Categorical(theta)
  end
  for n = 1:N
    w[n] ~ Categorical(phi[z[doc[n]]])
  end
  phi
end

(The full model can be found here)

If we introduce a vectorising notation, the likelihood computation step can be re-written as

@model nbmodel begin
  theta ~ Dirichlet(alpha)
  phi = Array{Any}(K)
  for k = 1:K
    phi[k] ~ Dirichlet(β)
  end
  z ~ Categorical(theta)                     #  z[m] ~ Categorical(theta)
  w ~ [Categorical(phi[z[doc[n]]) for n=1:n] # w[n] ~ Categorical(phi[z[doc[n]]])
  phi
end

The reason that vectorising code can be more efficient is due to improved automatic differentiation and inefficiency associated with loops.

Reference: Stan documentation (see the section 4.2)

[yebai]

Ref: #255

[xukai92]

Do we also want to support observations of multivariate distributions for data with nested-arrays?