-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generators using Stan models #35
Comments
Note that this is a specific generator usecase.
|
Hi, Not sure if this is the right section, but I was wondering about the possibility of generating data using the same Stan file that one would develop, for example, to run Prior Predictive checks (i.e.: data & generated quantities blocks only). I am asking because I have (and I expect many other users will have too) several such files already available and it would be nice to use them in SBC. Also, some are very complex, so converting them to a new generator would be quite a lot of work. Thanks! |
That is a great question. We have searched autogenerator from stanfile, but there were difficulties which @Dashadower could share further. One development option that comes to my mind is to use modular stan program which is the template for stanfile. If this modular program could be used to generate both SBC generator and stanfile (the latter @Dashadower is working on with the support from this) repo it would prevent an double effort. @alevaracca could you please share your thoughts on whether the above suggestion would meet your needs? Modular program is explained in more detail in the first section here. |
Thanks for the quick reply @hyunjimoon, I'll have a look into this and give it a try! I'll get back to you eventually. |
AFAIK, there are multiple ways people create their generators in Stan. To prioritise right: could you share what a typical (potentially simplified) Stan file you are using looks like? Are you using rstan or cmdstanr? (I'd be happy to get feedback on the implementation, so if you are willing to do some testing on the first version, I'll first implement a version that matches your needs). Note however, that a hard limitation of current Stan core is that there is no way to get to the results of the Also note that you can always create a dataset explicitly via Probably the most difficult conversion necessary is already covered by (currently undocumented) |
Just to chime in on using generated qualities for SBC; originally this was the way that the library was implemented. But it turned out that it was easier to extract the draws from a stanfit into a R object like rvar as to writing SBC for every model(what Martin is saying above). |
Thank you both for the reply. Martin, I'll give your approach a try in the next few days and get back with some feedback (plus some code of the model that I am SBC-ing for). |
Sorry if it took this much to get to this point, but I have followed Martin's instructions and they all worked out smoothly. Turns out it was not too complicated to tinker with the different formats. The conversion using |
I think my issue relates to this overall thread. I am working to impement SBC with the blavaan package, here we have pre compile Stan model, for usually large models with a lot of parameters. And within blavaan I can generate data sets from priors, so I could skip the generator function for example. But I cant include my list of data sets, as it is not an SBC_datasets type object. From this 2 questions and possible additions:
Appreciate any guidelines |
@maugavilla as this will likely require discussing a bunch of stuff that's specific to |
There are probably multiple flavors we could consider:
generated quantities
+ information on which parameters in the posterior are observablefixed_param
algorithm to run the generated quantities block which generates the observable valuesSBC_generator_brms
works)The text was updated successfully, but these errors were encountered: