-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compiler 3.0 #965
Compiler 3.0 #965
Conversation
I will review it tomorrow. |
Fixes #754. |
92c0a96
to
784b3ae
Compare
Please don't review this PR now except for the fun of it. I will break it down a bit. |
Could we add a macro |
Sure, I can do that. |
I wrote a summary of the features added in this PR. I will get to more tests and rebasing as soon as I can. |
So there seem to be some version conflicts. |
4add4d2
to
ce92abb
Compare
The tests are running properly now thanks to @goedman 's latest CmdStan release, thanks Rob! |
Many thanks, @mohamed82008! |
Mohamed, just to make sure, are you referring to CmdStan v5.3.0 or v5.3.1. I believe the latter as currently BinDeps and Homebrew are not version-wise compatible. |
5.3.1 yes |
Thank you. |
This WIP PR will include yet another refactor of the internals of Turing with the following main goals:
This PR currently implements the following features:
x[1]
andx[3]
as random variables andx[2]
andx[4]
as observations:missing
. This can be input when constructing the model or using the default value. If this missing variable is a vector and we need to loop or broadcast over it inside the model, the user needs to do:Note that any model argument whose value is not
missing
is not going to be handled as a random variable and will be treated as an observation. If an argument has no default value and is not input when constructing theTuring.Model
, an error will be thrown. Note that the aboveismissing
if-block is only needed if we need to loop overx
. If we only needx ~ MvNormal(...)
or something similar, then there is no need to initialize it whenismissing(x)
is true.DefaultContext
by default to calculate the log joint probability. But we can also run the model in different contexts to evaluate variants of thelogp
. For example, I defined theLikelihoodContext
which enables the computation of the log likelihood only instead of the log joint probability. Another context defined is theBatchContext
that computeslogprior + s * loglikelihood
, wheres = npoints / batch_size
. This is important for stochastic (batch-based) gradient algorithms to be optimizing the original objective in the expectation, and not highly weighing the prior.VarInfo
object in the model definition using the macro@varinfo()
. This is not a real macro and will throw if called outside the@model
definition. However, inside the@model
definition, one can use this to get a handle to theVarInfo
object inside the model run for more advanced use cases. There is also@logpdf()
which simply returns the current accumulated value of the log joint probability or whatever the context is having the model compute.Turing.Model
type and can be passed by the user using the special distribution typeNamedDist(dist, name)
. Sox ~ NamedDist(dist, name)
will have the random variable namedname
notx
.name
can be a string or aTuring.VarName
.VarName
constructor is now very neat and intuitive. You can dovn = Turing.@varname x[1,:][i]
to get aVarName{:x}
withvn.indexing == "[1,Colon()][$i]"
, whateveri
is.~
symbol is now supported for both observations and random variables. One can dox .~ dist
to mean that all elements ofx
are distributed according todist
. The main difference between~
and.~
in usage is that~
doesn't assume the left-hand-side to be defined while.~
does. So usingx .~ dist
whenx
wasn't previously defined will lead to an error. Broadcasting will work column-wise whendist isa MultivariateDistribution
andx isa AbstractMatrix
but notAbstractMatrix{>:AbstractVector}
. Ifx isa AbstractMatrix{Any}
for example, the broadcasting semantics here become ambiguous so we throw an error. This should not be an issue for most use cases.I still need to do more tests and rebase but I thought I should write this summary now because there will be no more features added to this PR, just fixes from now on.