James.yang/var vec mat

[JamesYang007]

TODO:

• Expression Checker moved to later issue
  • Generative Model Checking:
    • no variable gets assigned a distribution more than once
    • no variable gets assigned a distribution that references the same variable
    • no distribution references variables below it (ones that are assigned a distribution after current variable)
• Data/DataView with ppl::mat support
• ppl::dot
• extend fastad to support AD node for DotNode not needed!
• finish unit-testing everything that was temporarily given dummy tests
• test operator[] for vector-like param viewers
• refactor mcmc with new design
• update README
• cache AD expressions within some of the distribution expressions (see optimization)
  • potential issue: multiple chaining => multi-threading => race condition since AD expressions built by making copies of the underlying AD variables which will make them point to the original variable. Potential fix: make copies of model expressions for each thread and each copy has its own independent AD variables (do something in the copy constructor to make this happen).
• multiple chains/cores
• implement ESS: blog stan
• exceptions for non-activated models before calling inference models are pretty small, so inference can just activate.
• adaptive metropolis moved to later issue
• gibbs sampling moved to later issue

Optimization:

• sin(x+y) * cos(x+y) will recompute x+y twice. Think about having transformed parameters like STAN to do something like (w = x+y, y |= ppl::normal(w, 1.)); This optimization at the user-level that cannot be predicted or optimized from the library any further. User definitely has a say in how to optimize expressions.