estimating sigma_sq using an rbf kernel appears to be broken #10
Comments
bwpriest
added a commit
that referenced
this issue
Dec 9, 2021
* Addresses sigma_sq inference bug in issue #10 * Fixed test chassis to avoid overly smooth curves for to guarantee well-behaved estimators.
|
Fixed by PR #16 |
bwpriest
added a commit
that referenced
this issue
Dec 9, 2021
* Changed SigmaSq to its own class type. APIs no longer allow directly setting sigma_sq values, and instead only indicate whether to train them. * Update README.md missed dropping a reference to old sigma_sq api pattern * Hotfix/learn-sigma (#5) * Fixed a bug causing sigma_sqs to always go unlearned. * Changed default behavior throughout to automatically scale variance by sigma_sq (assuming it is trained). Overridable with the apply_sigma_sq kwarg. (#6) * Added option to all high-level apis to return crosswise_dists and pairwise_dists. (#7) * Added make_regress_tensors and make_train_tensors helper functions and dispersed them throughout the code. Simplified optimization API. Cleaned up some documentation. (#8) * Added CONTRIBUTING.md document. (#9) * Refactored BenchmarkGP to share new gp API. Added _sq_rel_err test function. Explored bug in rbf computation of sigma_sq, and turned off related test cases. See issue #1. (#11) * Feature/notebooks (#12) * First pass on a univariate regression tutorial, describing the basics of MuyGPyS. * Cleaned up some comments and documentation. * Linked tutorial notebook into docs and inserted links. * bugfix: changed return distances semantics and documentation to work correctly. * some miscellaneous cleanup. * Added regression API tutorial. * Edited readme to remove code and added links to notebooks. * Added matplotlib to docs/requirements.txt. (#13) * hopefully fixed rtd.yml (#14) * Docs (#15) * formatting fix to get readthedocs to build the project for notebooks * Hotfix/sigma sq approx (#16) * Addresses sigma_sq inference bug in issue #10 * Fixed test chassis to avoid overly smooth curves for to guarantee well-behaved estimators. * Fixed a bug in the sigma_sq testing chassis. (#18) * incremented minor version number.
bwpriest
added a commit
that referenced
this issue
Jan 19, 2022
* Changed SigmaSq to its own class type. APIs no longer allow directly setting sigma_sq values, and instead only indicate whether to train them. * Update README.md missed dropping a reference to old sigma_sq api pattern * Hotfix/learn-sigma (#5) * Fixed a bug causing sigma_sqs to always go unlearned. * Changed default behavior throughout to automatically scale variance by sigma_sq (assuming it is trained). Overridable with the apply_sigma_sq kwarg. (#6) * Added option to all high-level apis to return crosswise_dists and pairwise_dists. (#7) * Added make_regress_tensors and make_train_tensors helper functions and dispersed them throughout the code. Simplified optimization API. Cleaned up some documentation. (#8) * Added CONTRIBUTING.md document. (#9) * Refactored BenchmarkGP to share new gp API. Added _sq_rel_err test function. Explored bug in rbf computation of sigma_sq, and turned off related test cases. See issue #1. (#11) * Feature/notebooks (#12) * First pass on a univariate regression tutorial, describing the basics of MuyGPyS. * Cleaned up some comments and documentation. * Linked tutorial notebook into docs and inserted links. * bugfix: changed return distances semantics and documentation to work correctly. * some miscellaneous cleanup. * Added regression API tutorial. * Edited readme to remove code and added links to notebooks. * Added matplotlib to docs/requirements.txt. (#13) * hopefully fixed rtd.yml (#14) * Docs (#15) * formatting fix to get readthedocs to build the project for notebooks * Hotfix/sigma sq approx (#16) * Addresses sigma_sq inference bug in issue #10 * Fixed test chassis to avoid overly smooth curves for to guarantee well-behaved estimators. * Fixed a bug in the sigma_sq testing chassis. (#18) * incremented minor version number. * implemented a faster crosswise_distances function. (#19) * Modified training procedure to use pure functions. scipy_optimize_from_* functions now return an optimized model, leaving the model parameter object unaffected. (#21) This is a breaking update. loo_crossval signature changed significantly, and scipy_optimize_from_* functions now return an optimized MuyGPS object rather than modifying one in place. * Fixed an import bug introduced in last PR (#22) * feature/hyp_fix (#23) * Moved opt function preparation into KernelFn and MuyGPS classes to avoid yucky dict issues. * hyperparameter bounds no longer take the value "fixed". They instead default to (0.0, 0.0). The fixed status of hyperparameters is now accessed via Hyperparameter.fixed(). * SigmaSq no longer defaults to unlearned. Now defaults to [1.0], and added the SigmaSq.trained() boolean function to query if it has been set. * Added correct initialization checking to test chassis. * Small clean up. * Updated relevant documentation. * Relax scipy requirement to build on Python > 3.7 (#25) * Relax scipy requirement to build on Python > 3.7 Scipy 1.4.1 appears to be [incompatible with Python > 3.7][1]. Relaxing the exact version requirement makes it possible to install MuyGPyS on newer versions of Python. [1]: https://github.com/scipy/scipy/blob/adc4f4f7bab120ccfab9383aba272954a0a12fb0/setup.py#L44-L46 * incremented hnswlib version to 0.6.0. Co-authored-by: Ben Priest <bwpriester@gmail.com> * Adding more functional indirection to streamline optimization (#26) * Broke Matern static functions out into different versions for each nu setting to avoid boolean checks at each evaluation during optimization. * streamlined the loo_crossval objective function by wrapping it in a caller that only accepts the arguments that change. * Updated README to point to stable version of documentation. * Removed all outputs and timings from univariate regression tutorial. Co-authored-by: Imène Goumiri <imene.goumiri@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Trying to compute
sigma_sqviaMuyGPS.get_sigma_optimandget_analytic_sigma(forBenchmarkGP) appears to not be working, and will produce wrong results when trying to find a knownsigma_sqvalue. See the test casesGPSigmaSqBaselineTestandGPSigmaSqOptimTestinside oftests/optimize.py. Using"l2"rather than"F2"to create the pairwise distances appears to solve the issue in the latter case, but I am at a loss as to why.The text was updated successfully, but these errors were encountered: