Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream sampler results to disk (or other output) - minor feature suggestion #107

Open
bratslavia opened this issue Dec 31, 2019 · 1 comment

Comments

@bratslavia
Copy link

Something I often find useful when writing my own samplers is to write the output directly to disk in a streaming (on-line) fashion.

It might sound silly, but I've worked in places where forced system updates/reboots have lost me days worth of work at a time, and saving sampler output directly to disk has let me at least recover some of the work (and use it, e.g., for helping to determine potential convergence issues with new/complex models, as prep for subsequent adaptive MCMC runs, etc.) There are also cases where piping the sampler output to some other program in real time (e.g., for plotting, tracking particular parameters of interest, etc.) has been helpful (a slightly different, but related, issue to piping to disk).

@cpfiffer
Copy link
Member

cpfiffer commented Jan 2, 2020

I think we could probably do this by adding generic callback functions to sampling, such that you could have some kind of a plotting/saving/reporting/doing stuff functionality. This exists in one of my branches but it has not yet been formalized into good code -- I can add this in, because I think it's important functionality to have.

Currently this would take place upstream in AbstractMCMC, which I'm more than happy to take a look at.

As to the form that the storage should take, I think probably we could just save each Transition struct in a JLD2 file, and provide a little bit of code to allow for a quick resume of a chain that was terminated because of an error or something.

@yebai yebai transferred this issue from TuringLang/Turing.jl Nov 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants