Feature/issue 1426 ibeta derivs large args #1433

betanalpha · 2015-04-29T21:23:08Z

Summary:

Dramatically improves the robustness and performance of the regularized incomplete beta function derivatives, propagating the chances to neg_binomial_cdf and neg_binomial_2_cdf. Changes still need to be propagated to beta_cdf, student_t_cdf, and others.

Intended Effect:

neg_binomial_cdf and neg_binomial_2_cdf gradients are now accurate for large arguments.

How to Verify:

./runTests.py src/test/prob/neg_binomial
./runTests.py src/test/prob/neg_binomial_2

Or build a crazy model using neg_binomial_cdf or neg_binomial_2_cdf with large arguments.

Side Effects:

None. Performance should actually improve with the added checks that switch to different expansions for different arguments.

Documentation:

None.

Reviewer Suggestions:

None.

betanalpha · 2015-04-29T23:14:24Z

@Syklic Are these fine? I ran cpplint on the new/modified stuff before
the pull request and it passed (or rather I deleted a lot of
automatically-indented whitepspace until it passed).

On Apr 30, 2015, at 12:03 AM, Stan buildbot notifications@github.com wrote:

Refer to this link for build results (access rights to CI server needed):
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/352/

Build Log
last 20 lines

[...truncated 16 lines...]
Starting build job Stan Pull Request - Doc - Doxygen.
Starting build job Stan Pull Request - Doc - Reference Manual.
Finished Build : #346 of Job : Stan Pull Request - Doc - Reference Manual with status : SUCCESS
Finished Build : #352 of Job : Stan Pull Request - Doc - Doxygen with status : SUCCESS
Starting build job Stan Pull Request - Tests - Unit.
Starting build job Stan Pull Request - Tests - Integration.
Starting build job Stan Pull Request - Windows - Tests - Unit.
Starting build job Stan Pull Request - Tests - Header.
Finished Build : #318 of Job : Stan Pull Request - Tests - Header with status : SUCCESS
Finished Build : #320 of Job : Stan Pull Request - Tests - Integration with status : SUCCESS
Finished Build : #166 of Job : Stan Pull Request - Windows - Tests - Unit with status : SUCCESS
Finished Build : #320 of Job : Stan Pull Request - Tests - Unit with status : SUCCESS
Starting build job Stan Pull Request - CppLint.
Starting build job Stan Pull Request - Math Dependencies.
Finished Build : #72 of Job : Stan Pull Request - Math Dependencies with status : SUCCESS
Finished Build : #113 of Job : Stan Pull Request - CppLint with status : FAILURE
Build step 'MultiJob Phase' marked build as failure
Warning: you have no plugins providing access control for builds, so falling back to legacy behavior of permitting any downstream builds to be triggered
Setting status of 7e8db1c to FAILURE with url http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/352/ and message: Merged build finished.
Test FAILed.

—
Reply to this email directly or view it on GitHub.

syclik · 2015-05-04T15:09:00Z

@betanalpha, could you add some basic tests for:

ddz_inc_beta
ddb_inc_beta
dda_inc_beta

that can go in src/test/unit/math/prim/scal/fun/inc_beta_derivatives.hpp for now. All we would need is a double-based test to go there, unless this is used for stan::agrad::var also.

Or, you can just give me reasonable values for each of the values just so we can instantiate these functions stand-alone and get reasonable values out.

I'll add some more comments in the code itself.

…github.com/stan-dev/stan into feature/issue-1426-ibeta_derivs_large_args Conflicts: src/stan/math/prim/scal/fun/inc_beta_derivatives.hpp

betanalpha · 2015-05-05T12:25:43Z

Double test added.

On May 4, 2015, at 4:09 PM, Daniel Lee notifications@github.com wrote:

@betanalpha, could you add some basic tests for:

ddz_inc_beta
ddb_inc_beta
dda_inc_beta
that can go in src/test/unit/math/prim/scal/fun/inc_beta_derivatives.hpp for now. All we would need is a double-based test to go there, unless this is used for stan::agrad::var also.

Or, you can just give me reasonable values for each of the values just so we can instantiate these functions stand-alone and get reasonable values out.

I'll add some more comments in the code itself.

—
Reply to this email directly or view it on GitHub.

syclik · 2015-05-05T15:19:20Z

Final needs (I'll take care of it):

split apart functions to own files
split apart tests appropriately

…426-ibeta_derivs_large_args

syclik · 2015-05-11T11:53:44Z

@betanalpha: the functions are missing documentation. I'll start filling stuff in, but might need your help. The incomplete beta derivatives have been renamed and moved to:

src/stan/math/prim/scal/fun/inc_beta_dda.hpp
src/stan/math/prim/scal/fun/inc_beta_ddb.hpp
src/stan/math/prim/scal/fun/inc_beta_ddz.hpp

betanalpha · 2015-05-11T12:04:16Z

Gimme an example of what you want.

On May 11, 2015, at 12:53 PM, Daniel Lee notifications@github.com wrote:

@betanalpha: the functions are missing documentation. I'll start filling stuff in, but might need your help. The incomplete beta derivatives have been renamed and moved to:

src/stan/math/prim/scal/fun/inc_beta_dda.hpp
src/stan/math/prim/scal/fun/inc_beta_ddb.hpp
src/stan/math/prim/scal/fun/inc_beta_ddz.hpp
—
Reply to this email directly or view it on GitHub.

syclik · 2015-05-22T14:09:51Z

@betanalpha, I added some doc, but I feel like it's wrong. Feel like taking a look? What I really wanted to add was something that indicated where it is expected not to work.

stan-buildbot · 2015-05-22T14:16:10Z

Refer to this link for build results (access rights to CI server needed):
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/383/

Build Log
last 20 lines

[...truncated 11 lines...]
 > /usr/local/git/bin/git rev-parse refs/remotes/origin/origin/pr/1433/merge^{commit} # timeout=10
Checking out Revision a7d88f4780728691ecfa1506969dd6e5b3361492 (refs/remotes/origin/pr/1433/merge)
 > /usr/local/git/bin/git config core.sparsecheckout # timeout=10
 > /usr/local/git/bin/git checkout -f a7d88f4780728691ecfa1506969dd6e5b3361492
First time build. Skipping changelog.
Starting build job Stan Pull Request - Doc - Reference Manual.
Starting build job Stan Pull Request - Doc - Doxygen.
Finished Build : #377 of Job : Stan Pull Request - Doc - Reference Manual with status : SUCCESS
Finished Build : #383 of Job : Stan Pull Request - Doc - Doxygen with status : SUCCESS
Starting build job Stan Pull Request - Tests - Integration.
Starting build job Stan Pull Request - Tests - Header.
Starting build job Stan Pull Request - Windows - Tests - Unit.
Starting build job Stan Pull Request - Tests - Unit.
Finished Build : #349 of Job : Stan Pull Request - Tests - Header with status : FAILURE
Finished Build : #351 of Job : Stan Pull Request - Tests - Integration with status : ABORTED
Finished Build : #351 of Job : Stan Pull Request - Tests - Unit with status : ABORTED
Finished Build : #198 of Job : Stan Pull Request - Windows - Tests - Unit with status : ABORTED
Build step 'MultiJob Phase' marked build as failure
Setting status of 4ae4f5e020518cf670e89a6d23f6df5078cf03ff to FAILURE with url http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/383/ and message: Build finished.

Test FAILed.

betanalpha · 2015-05-23T15:14:36Z

How's that?

stan-buildbot · 2015-06-01T18:10:47Z

Refer to this link for build results (access rights to CI server needed):
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/396/

Build Log
last 20 lines

[...truncated 11 lines...]
 > /usr/local/git/bin/git rev-parse refs/remotes/origin/origin/pr/1433/merge^{commit} # timeout=10
Checking out Revision 11d2c3dd8cb6aa01e29edbc3d9fce5dca4e86cd0 (refs/remotes/origin/pr/1433/merge)
 > /usr/local/git/bin/git config core.sparsecheckout # timeout=10
 > /usr/local/git/bin/git checkout -f 11d2c3dd8cb6aa01e29edbc3d9fce5dca4e86cd0
First time build. Skipping changelog.
Starting build job Stan Pull Request - Doc - Reference Manual.
Starting build job Stan Pull Request - Doc - Doxygen.
Finished Build : #390 of Job : Stan Pull Request - Doc - Reference Manual with status : SUCCESS
Finished Build : #396 of Job : Stan Pull Request - Doc - Doxygen with status : SUCCESS
Starting build job Stan Pull Request - Tests - Integration.
Starting build job Stan Pull Request - Tests - Unit.
Starting build job Stan Pull Request - Tests - Header.
Starting build job Stan Pull Request - Windows - Tests - Unit.
Finished Build : #362 of Job : Stan Pull Request - Tests - Header with status : FAILURE
Finished Build : #364 of Job : Stan Pull Request - Tests - Unit with status : ABORTED
Finished Build : #364 of Job : Stan Pull Request - Tests - Integration with status : ABORTED
Finished Build : #211 of Job : Stan Pull Request - Windows - Tests - Unit with status : ABORTED
Build step 'MultiJob Phase' marked build as failure
Setting status of 1c63764a1ca560e3a02d7c63cf8a1eebf19c7c80 to FAILURE with url http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/396/ and message: Build finished.

Test FAILed.

stan-buildbot · 2015-06-01T18:43:15Z

Refer to this link for build results (access rights to CI server needed):
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/397/

Build Log
last 20 lines

[...truncated 11 lines...]
 > /usr/local/git/bin/git rev-parse refs/remotes/origin/origin/pr/1433/merge^{commit} # timeout=10
Checking out Revision 41345ae3491996e5c0bc108a785bcc6264c3fbd6 (refs/remotes/origin/pr/1433/merge)
 > /usr/local/git/bin/git config core.sparsecheckout # timeout=10
 > /usr/local/git/bin/git checkout -f 41345ae3491996e5c0bc108a785bcc6264c3fbd6
First time build. Skipping changelog.
Starting build job Stan Pull Request - Doc - Reference Manual.
Starting build job Stan Pull Request - Doc - Doxygen.
Finished Build : #391 of Job : Stan Pull Request - Doc - Reference Manual with status : SUCCESS
Finished Build : #397 of Job : Stan Pull Request - Doc - Doxygen with status : SUCCESS
Starting build job Stan Pull Request - Windows - Tests - Unit.
Starting build job Stan Pull Request - Tests - Integration.
Starting build job Stan Pull Request - Tests - Unit.
Starting build job Stan Pull Request - Tests - Header.
Finished Build : #363 of Job : Stan Pull Request - Tests - Header with status : SUCCESS
Finished Build : #365 of Job : Stan Pull Request - Tests - Integration with status : FAILURE
Finished Build : #365 of Job : Stan Pull Request - Tests - Unit with status : ABORTED
Finished Build : #212 of Job : Stan Pull Request - Windows - Tests - Unit with status : ABORTED
Build step 'MultiJob Phase' marked build as failure
Setting status of 836d3355ef197180598936ca933090d3c8ceebb3 to FAILURE with url http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/397/ and message: Build finished.

Test FAILed.

stan-buildbot · 2015-06-02T07:08:00Z

Refer to this link for build results (access rights to CI server needed):
http://d1m1s1b1.stat.columbia.edu:8080/job/Stan%20Pull%20Request/399/
Test PASSed.

…s_large_args fixes #1426. Feature/issue 1426 ibeta derivs large args

betanalpha added 5 commits April 26, 2015 22:26

More robust regularized ibeta derivatives and negative binomial cdfs

b465d12

Working for doubles, namespace issues for higher-order

eece634

Substantially more robust regularized incomplete beta derivatives

7ef2c9f

Fixed incorrect function name

cfa4dc7

Included what you use - test headers now passes

7e8db1c

removed more cpplint warnings

64a453a

betanalpha added 2 commits May 5, 2015 13:21

Added more stability for large values of second argument, test

6b61d1e

Merge branch 'feature/issue-1426-ibeta_derivs_large_args' of https://…

08e8dea

…github.com/stan-dev/stan into feature/issue-1426-ibeta_derivs_large_args Conflicts: src/stan/math/prim/scal/fun/inc_beta_derivatives.hpp

cpplint warnings

a71514a

syclik added 3 commits May 8, 2015 23:01

Merge branch 'feature/issue-1441-math-namespace' into feature/issue-1…

fded172

…426-ibeta_derivs_large_args

1426: changing function names

4281233

splitting out inc_beta derivatives

46132e5

basic doc

4ae4f5e

Doc for inc_beta derivatives

cb6de77

syclik added 2 commits May 27, 2015 17:03

updating doc for ibeta deriv

c38b669

Merge branch 'develop' into feature/issue-1426-ibeta_derivs_large_args

4e98ace

syclik added this to the v2.6.3++ milestone May 27, 2015

syclik added the code review complete label May 27, 2015

syclik added 3 commits May 27, 2015 17:33

fixing includes

4411139

Merge branch 'develop' into feature/issue-1426-ibeta_derivs_large_args

5deae91

1426: my bad -- mismatched curly brace

e101fb2

1426: fixing merge problem

1c63764

1426: fixing merge problem

836d335

1426: last bit of renaming

229c283

syclik added a commit that referenced this pull request Jun 2, 2015

Merge pull request #1433 from stan-dev/feature/issue-1426-ibeta_deriv…

ee7538b

…s_large_args fixes #1426. Feature/issue 1426 ibeta derivs large args

syclik merged commit ee7538b into develop Jun 2, 2015

syclik deleted the feature/issue-1426-ibeta_derivs_large_args branch June 2, 2015 18:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/issue 1426 ibeta derivs large args #1433

Feature/issue 1426 ibeta derivs large args #1433

betanalpha commented Apr 29, 2015

betanalpha commented Apr 29, 2015

syclik commented May 4, 2015

betanalpha commented May 5, 2015

syclik commented May 5, 2015

syclik commented May 11, 2015

betanalpha commented May 11, 2015

syclik commented May 22, 2015

stan-buildbot commented May 22, 2015

betanalpha commented May 23, 2015

stan-buildbot commented Jun 1, 2015

stan-buildbot commented Jun 1, 2015

stan-buildbot commented Jun 2, 2015

Feature/issue 1426 ibeta derivs large args #1433

Feature/issue 1426 ibeta derivs large args #1433

Conversation

betanalpha commented Apr 29, 2015

Summary:

Intended Effect:

How to Verify:

Side Effects:

Documentation:

Reviewer Suggestions:

betanalpha commented Apr 29, 2015

syclik commented May 4, 2015

betanalpha commented May 5, 2015

syclik commented May 5, 2015

syclik commented May 11, 2015

betanalpha commented May 11, 2015

syclik commented May 22, 2015

stan-buildbot commented May 22, 2015

betanalpha commented May 23, 2015

stan-buildbot commented Jun 1, 2015

stan-buildbot commented Jun 1, 2015

stan-buildbot commented Jun 2, 2015