New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analysis API: compute effective sample size #2569

Open
roualdes opened this Issue Jul 5, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@roualdes
Contributor

roualdes commented Jul 5, 2018

Summary:

Offer interfaces a unified calculation of effective sample size.

Description:

This issue is solely for the calculation of ess for a single parameter across multiple chains.

Many details were discussed on the Discourse: Analysis API thread. Design details:

  • route goes in stan-dev/stan:src/stan/analyze/mcmc/compute_ess.hpp
  • signature is compute_effective_sample_size(std::vector<double*> draws, std::vector<size_t> sizes) where each chain's draws are stored in a contiguous block of memory
  • convenience function compute_effective_sample_size(std::vector<double*> draws, size_t size)
  • calculation and I/O handled separately. Only calculation, here.
  • minimize copying
  • stan-dev/stan is the appropriate place for ess calculation (and other diagnostics) as it is more closely related to sampling algorithms than the stan-dev/math library
  • start with ess, integrate it all the way through, then build out other diagnostics
  • long term goal to remove ess from chains class as OO structure is not necessary

Current Version:

v2.17.1

@roualdes

This comment has been minimized.

Contributor

roualdes commented Jul 5, 2018

Looking at effective_sample_size within the chains class, will n_kept_samples ever be different from n_samples? If so, do we need to accept n_kept_samples as an argument to compute_effective_sample_size?

@bob-carpenter

This comment has been minimized.

Contributor

bob-carpenter commented Jul 6, 2018

@riddell-stan

This comment has been minimized.

riddell-stan commented Jul 6, 2018

@roualdes

This comment has been minimized.

Contributor

roualdes commented Jul 9, 2018

I believe the answer to my question is, no, n_kept_samples and n_samples will never be different. My best efforts indicate that develop:effective_sample_size is only ever given kept draws, as @bob-carpenter suggested.

PR #2575 is now posted. I took some liberties on the code, and tried to justify them in the post.

Apologies for jumping this conversation across three different pages: Discourse, this issue, and the above PR.

@kedartal

This comment has been minimized.

Contributor

kedartal commented Aug 8, 2018

I hope this is the right place for this issue, apologies in advance if not.
I get a "duplicate symbols" linker error for the functions in .../stan/analyze/mcmc/compute_effective_sample_size.hpp when compiling against Stan (commit 11b1c9c); adding "inline" to the signatures solves the issue. Would it make sense to solve it this way, or alternatively, move the code into a cpp file?

@bob-carpenter

This comment has been minimized.

Contributor

bob-carpenter commented Aug 9, 2018

@kedartal: This is not the right place. This is a specific issue. I don't want to hijack this issue with a discussion of inline.

If you have a reproducible example of a bug, you can create a new issue. Otherwise, we have a forum for general discussion if you have a question:

http://discourse.mc-stan.org

@roualdes

This comment has been minimized.

Contributor

roualdes commented Aug 22, 2018

Can this be closed now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment