Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/improve nuts diagnostic #352

Merged
merged 6 commits into from
Nov 26, 2013
Merged

Conversation

betanalpha
Copy link
Contributor

Introduced a new NUTS statistic that helps to identify over-aggressive adaptation (and tweaked the acceptance stat to use all samples in the NUTS tree instead of just those in the last subtree). Documentation updated and test-unit passes.

I know it may not be possible to modify the Jenkins queue all that easily, but the changes are relevant for some of the problems users are seeing and might be worthy of elevation to higher priority if deemed appropriate.

…ions for diagnosing adaptive HMC implementations
Conflicts:
	src/stan/mcmc/hmc/nuts/base_nuts.hpp
@stan-buildbot
Copy link
Contributor

  • code review

    Can one of the admins verify this patch?

@betanalpha
Copy link
Contributor Author

How do we confirm compatibility with the R and Python interfaces? The updates here are minimal but can make a big difference in diagnostics, especially when trying to help diagnose problems remotely so I'd like to get any discussion going as quickly as possible.

@bob-carpenter
Copy link
Contributor

I was only suggesting sending mail to Allen and Jiqiang!

  • Bob

On 11/6/13, 6:33 AM, Michael Betancourt wrote:

How do we confirm compatibility with the R and Python interfaces? The updates here are minimal but can make a big
difference in diagnostics, especially when trying to help diagnose problems remotely so I'd like to get any discussion
going as quickly as possible.

@betanalpha
Copy link
Contributor Author

Allen has confirmed that PyStan just dumps the output stream to text and does not manipulate
any of the fields directly and so isn't effected by new sampler diagnostics. Haven't heard back
from Jiqiang.

On Nov 6, 2013, at 6:44 PM, Bob Carpenter notifications@github.com wrote:

I was only suggesting sending mail to Allen and Jiqiang!

  • Bob

On 11/6/13, 6:33 AM, Michael Betancourt wrote:

How do we confirm compatibility with the R and Python interfaces? The updates here are minimal but can make a big
difference in diagnostics, especially when trying to help diagnose problems remotely so I'd like to get any discussion
going as quickly as possible.


Reply to this email directly or view it on GitHub.

@maverickg
Copy link
Contributor

Looks fine as it does not change command.hpp.

@syclik
Copy link
Member

syclik commented Nov 11, 2013

Jenkins, ok to test.

@@ -2147,8 +2147,9 @@ \section{Running the Print Command}
accept_stat__ 0.88 5.6e-03 1.8e-01 0.51 0.95 1.0 1000 16881 1.00
stepsize__ 0.30 1.3e-15 8.9e-16 0.30 0.30 0.30 0.50 8.5 1.00
treedepth__ 1.4 2.6e-02 8.0e-01 0.00 1.0 2.0 946 15978 1.00
n_divergent__ 1.4 0.0e+00 0.0e+00 0.00 0.0 0.0 1000 16949 1.00
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be n_divergence__?

@syclik
Copy link
Member

syclik commented Nov 12, 2013

@betanalpha, looks like all the model tests are going to fail since the n_divergent__ parameter wasn't being ignored in all the tests. I can handle it later this week, if you don't get to it first.

@betanalpha
Copy link
Contributor Author

I forgot model tests were back in the fold. I won't be able to get to it until Thursday (another talk tomorrow) but if you haven't touched it by then
I'll tackle it.

On Nov 12, 2013, at 9:45 PM, Daniel Lee notifications@github.com wrote:

@betanalpha, looks like all the model tests are going to fail since the n_divergent__ parameter wasn't being ignored in all the tests. I can handle it later this week, if you don't get to it first.


Reply to this email directly or view it on GitHub.

@betanalpha
Copy link
Contributor Author

One character fix. Model tests look good on my machine now.

On Tue, Nov 12, 2013 at 9:49 PM, Michael Betancourt betanalpha@gmail.comwrote:

I forgot model tests were back in the fold. I won't be able to get to it
until Thursday (another talk tomorrow) but if you haven't touched it by then
I'll tackle it.

On Nov 12, 2013, at 9:45 PM, Daniel Lee notifications@github.com wrote:

@betanalpha, looks like all the model tests are going to fail since the
n_divergent__ parameter wasn't being ignored in all the tests. I can handle
it later this week, if you don't get to it first.


Reply to this email directly or view it on GitHub.

@stan-buildbot
Copy link
Contributor

Test FAILed.
Refer to this link for build results: http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Github%20Pull%20Requests/281/

@syclik
Copy link
Member

syclik commented Nov 21, 2013

I stopped the build. I'll get it back on after the one currently being
tested, which cleans up all the compiler warnings.

On Thu, Nov 21, 2013 at 11:08 AM, Stan buildbot notifications@github.comwrote:

Test FAILed.
Refer to this link for build results:
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Github%20Pull%20Requests/281/


Reply to this email directly or view it on GitHubhttps://github.com//pull/352#issuecomment-28997145
.

@syclik
Copy link
Member

syclik commented Nov 25, 2013

Jenkins, retest this please.

@stan-buildbot
Copy link
Contributor

Test FAILed.
Refer to this link for build results: http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Github%20Pull%20Requests/290/

@syclik
Copy link
Member

syclik commented Nov 26, 2013

Jenkins, retest this please.

@stan-buildbot
Copy link
Contributor

Test PASSed.
Refer to this link for build results: http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Github%20Pull%20Requests/291/

syclik added a commit that referenced this pull request Nov 26, 2013
@syclik syclik merged commit 9f2f776 into develop Nov 26, 2013
@syclik syclik deleted the feature/improve_nuts_diagnostic branch November 26, 2013 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants