Skip to content

Value-based tests for log-likelihoods#728

Merged
MichaelClerx merged 12 commits intomasterfrom
i719-log-likelihood-tests
Feb 22, 2019
Merged

Value-based tests for log-likelihoods#728
MichaelClerx merged 12 commits intomasterfrom
i719-log-likelihood-tests

Conversation

@ben18785
Copy link
Copy Markdown
Collaborator

@ben18785 ben18785 commented Feb 19, 2019

In this checking of the tests, I (somewhat disconcertingly) found about a dozen errors that would have been picked up if we were doing value-based tests of log-likelihoods. The errors were on the following:

  • AR1 values for multiple output problems
  • ARMA11 values for multiple output problems
  • Gaussian log likelihood values and derivatives for multiple output problems. Given that this is our workhorse distribution for many problems this was particularly concerning
  • The ConstantModel which we use for testing previously had incorrect derivatives; these should always be 1 since f = theta -> df / dtheta = 1.

Covers #719

fcooper8472 and others added 10 commits February 19, 2019 15:05
- was previously pulling out the wrong sigma and rho for multi output problems
- Added multi-output test for ARMA11
- Corrected parameter referencing in ARMA11 code
- Corrected docs for ARMA11 code
- The sensitivities were previously not correct for multi-output problems
- Added multi-output value-based tests for multi-output problems
- Corrected mistake with constant_model forward sensitivities (it should be a diagonal matrix rather than one with increasing diagonals)
- Corrected errors with sensitivities for Gaussian log-likelihood for multiple output problems: formerly we were returning an array of theta derivatives followed by a single sigma derivative. This is incorrect as there are three sigma parameters - one for each output
- Added value-based tests for Gaussian with known sigma
@ben18785 ben18785 changed the title Value-based tests for log-likelihood Value-based tests for log-likelihoods Feb 19, 2019
- constant model test was wrong previously
- corrected incorrect derivative-based tests for the constant model
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 19, 2019

Codecov Report

Merging #728 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff          @@
##           master   #728   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files          50     50           
  Lines        4622   4628    +6     
=====================================
+ Hits         4622   4628    +6
Impacted Files Coverage Δ
pints/_log_likelihoods.py 100% <100%> (ø) ⬆️
pints/toy/_constant_model.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a08b15...7e9edce. Read the comment docs.

Copy link
Copy Markdown
Member

@MichaelClerx MichaelClerx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Ben, Thanks very much! Some tiny suggestions and then happy to merge

Comment thread pints/toy/_constant_model.py Outdated
- tweak to docstring for constant model
- tweak to derivative function to simplify for constant model
- tweak to test comments for error measures to reflect changes to the derivative function for the constant model
Copy link
Copy Markdown
Member

@fcooper8472 fcooper8472 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@ben18785
Copy link
Copy Markdown
Collaborator Author

@MichaelClerx Can we merge this please if you're happy with my changes?

@MichaelClerx MichaelClerx merged commit cc6562f into master Feb 22, 2019
@MichaelClerx MichaelClerx deleted the i719-log-likelihood-tests branch February 22, 2019 10:18
MichaelClerx added a commit that referenced this pull request Feb 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants