Don't include -np.inf in calculating average ELBO
#1880
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
twiecki
reviewed
Mar 7, 2017
pymc3/variational/advi.py
Outdated
|
|
||
|
|
||
| def infmean(input_array): | ||
| """Return the median of the finite values of the array""" |
There was a problem hiding this comment.
Good catch, my mistake.
(Meant to skip ci on the fix, sadly looks like I can't cancel it after the fact.)
|
Thanks @pstjohn! |
davidbrochart
pushed a commit
to davidbrochart/pymc3
that referenced
this pull request
Mar 27, 2017
* Adds an infmean for advi reporting * fixing typo
twiecki
pushed a commit
that referenced
this pull request
Mar 27, 2017
* Added live_traceplot function * Cosmetic change * Changed the API to pm.sample(..., live_plot=True) * Don't include `-np.inf` in calculating average ELBO (#1880) * Adds an infmean for advi reporting * fixing typo * Add tutorial to detect sampling problems (#1866) * Expand sampler-stats.ipynb example include model diagnose from case study example in Stan http://mc-stan.org/documentation/case-studies/divergences_and_bias.html * Sampler Diagnose for NUTS * descriptive annotation and axis labels * Fix typos * PEP8 styling * minor updates 1, add example to examples.rst 2, original content in Markdown code block * Make install scripts idempotent (#1879) * DOC Change heading names. * Add examples of censored data models (#1870) * Raise TypeError on non-data values of observed (#1872) * Raise TypeError on non-data values of observed * Added check for observed TypeError * Make exponential mode have the correct shape * Fix support of LKJCorr * Added tutorial notebook on updating priors * Fixed y-axis bug in forestplot; added transform argument to summary * Style cleanup * Made small changes and executed the notebook * Added probit and invprobit functions * Added carriage return to end of file * Fixed indentation * Changed probit test to use assert_allclose * Fix tests for LKJCorr * Added warning for ignoring init arguments in sample * Kill stray tab * Improve performance of transformations * DOC Add new features * Bump version. * Added docs and scripts to MANIFEST * WIP: Implement opvi (#1694) * migrate useful functions from previous PR (cherry picked from commit 9f61ab4) * opvi draft (cherry picked from commit d0997ff) * made some test work (cherry picked from commit b1a87d5) * refactored approximation to support aevb (without test) * refactor opvi delete unnecessary methods from operator, change method order * change log_q_local computation * add full rank approximation * add more_params argument to ObjectiveFunction.updates (aevb case) * refactor density computation in full rank approximation * typo: cast dict values to list * typo: cast dict values to list * typo: undefined T in dist_math * refactor gradient scaling as suggested in approximateinference.org/accepted/RoederEtAl2016.pdf * implement Langevin-Stein (LS) operator * fix docstring * add blank line in docs * refactor ObjectiveFunction * add not working LS Op test * experiments with not working LS Op * change activations * refactor networks * add step_function * remove Langevin Stein, done refactoring * remove Langevin Stein, done refactoring * change optimizers * refactor init params * implement tests * implement Inference * code style * test fix * add minibatch test (fails now) * add more tests for minibatch training * add logdet to FullRank approximation * add conversion of arrays to floatX * tiny changes * change number of iterations * fix test and pylint check * memoize functions in Objective function * Optimize code a lot * a bit more efficient pickling * add docs * Add MeanField -> FullRank parameter transfer * refactor MeanField and FullRank a bit * fix FullRank bug with shapes in random * refactor Model.flatten (CC @taku-y) * add `approximate` to inference * rename approximate->fit * change abbreviations * Fix bug with scaling input variable in aevb * fix theano bottleneck in graph * more efficient scaling for local vars * fix typo in local Q * add aevb test * refactor memoize to work with my objects * add tests for numpy view usage * pickle-hash fix * pickle-hash fix again * add node sampling + make up some code * add notebook with example * sample_proba explained * Revert "small fix for multivariate mixture models" * Added message about init only working with auto-assigned step methods * doc(DiagInferDiv): formatting fix in blog post quote. Closes #1895. (#1909) * delete unnecessary text and add some benchmarks (#1901) * Add LKJCholeskyCov * Added newline to MANIFEST * Replaced package list with find_packages in setup.py; removed examples/data/__init__.py * Fix log jacobian in LKJCholeskyCov * Updated version to rc2 * Fixed stray version string * Fix indexing traces with steps greater one * refactor variational module, add histogram approximation (#1904) * refactor module, add histogram * add more tests * refactor some code concerning AEVB histogram * fix test for histogram * use mean as deterministic point in Histogram * remove unused import * change names of shortcuts * add names to shared params * add new line at the end of `approximations.py` * Add documentation for LKJCholeskyCov * SVGD problems (#1916) * fix some svgd problems * switch -> ifelse * except in record * Histogram docs (#1914) * add docs * delete redundant code * add usage example * remove unused import * Add expand_packed_triangular * improve aesthetics * Bump theano to 0.9.0rc4 (#1921) * Add tests for LKJCholeskyCov * Histogram: use only free RVs from trace (#1926) * use only free RVs from trace * use memoize in Histogram.histogram_logp * Change tests for histogram * Bump theano to be at least 0.9.0 * small fix to prevent a TypeError with the ufunc true_divide * Fix tests for py2 * Add floatX wrappers in test_advi * Changed the API to pm.sample(..., live_plot=True) * Better formatting
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses a minor issue in which likelihoods which can return a 0 probability (-inf log-probability) often report a negative infinity average ELBO for the entire simulation.
For instance,
Here, I just report the mean after removing the -np.inf values so convergence can be monitored.