Throw exception if model metrics are incomplete #456
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… minor style updates
* move viewer to within whylogs * add profile viewer * 📚 add documentation for viewer * edits on profile viewer documentation * remove dividers * typos,and loom gif * use thumbnail gif from loom * Update README.md Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> * move metric collection list * Remove dots * Updating whylogs demo gif * Update README.md * Update README.md * Update README.md * bump version 0.4.8 Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> Co-authored-by: Sam Gracie <4944259+samgracie@users.noreply.github.com>
Co-authored-by: Chris Warth <csw@whylabs.ai>
Co-authored-by: Andy Dang🤖 <andy@whylabs.ai>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Don't include secrets
* Fix KS test p-value computation, add condition in entropy * Change number of samples calculation Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
* Fix typos and add example for the missing values proportion constraint in Constraints_Suite Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: MilenaTrajanoska <trajanoska28@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
cswarth
changed the title
Pull Request Test Coverage Report for Build 1897004716Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
* Add feast integration demo * - Removed parquet files from the repo and moved to s3 * add .gitkeep to push empty data folder
Added logo, cleaned up intro text, reorganized some sections
Cleaned up typos, added Contributors subsection, centered badges, added a section describing what can be done with profiles
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
* Fix missing dataTimestamp post aggregation * Add a scala test to cover deserializing the dataTimestamp from the DatasetProfile
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* add performance metrics - classification and regression - notebooks
* Matrix build for Spark and Scala versions * add spark 3.1.1 * publish matrix builds * include Spark 3.2 & Scala 2.13 * need os in included matrix entry * Scala 2.13 will need code changes * match publish matrix to build matrix * spark/scala default versions Co-authored-by: Richard Rogers <rrogers@opendius.net> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
* visualizer new design, add reference profile in profile viewer * add json reference profile button, add UI changes * change view detail button position, change referenceProfilePanel chart view, add icons * add property panel for reference profile, charts for json profile without reference * add the notification sign of filter button * use addFileName() function when the file is load * add "Filter by type" design changes, add "Quick search" design changes. * add "Quick search" input. * count of types in "filter by type" dropdown * add double histogram chart in reference profile panel. * "Sign up for a free WhyLabs account" button disappears when screen with less than 1000px. * add reference profile charts, add profile name and time stamp * add diff from ref when reference profile consists. * Shows alert when the key of profile data is not matched to reference profile data. * histograms visualization in jupyter notebook * add backend for jupyter notebook vizualisation, add jupyter notebook to display profile data in the notebook * add double histogram visualization * responsive distribution chart * add search logic * add folder for saved htmls, add all methods to jupyter notebook cells, change backend code stucture * add frequent items in reference property panel * add feature summary statistics backend, add single profile summary statistics backend * add Full Profile Summary Statistics visualization * add Feature Summary Statistics visualisation * add ability to add multiple features in double histogram. * show full number at hover on statistics numbers * add documentation in notebook, naming fixes * add calculations in backend for variance and coefficient of variation * add constraints report backend, add constraints report in notebook * constraints report with real values * Categorical drift report: Differenced Bar chart jupyter visualization * Add definition (ranges) for the severe, moderate, low drift categories. * change import in tests, add categorical profile generator in tests * change css and js paths for browser visualizer * Histogram Bin Width and Range (#468) - Changed upper value of domain's xscale to match the max between both distribution's -> max of bin value + bin_width - apply Xscale to each of distribution's [1] and [0] before calculating bar width, instead of scaling the difference ([1]-[0]) Co-authored-by: Perch <perchgithub@gmail.com> Co-authored-by: ssxcho <xchton@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com> Co-authored-by: Khachatur Tonoyan <31924981+ssxcho@users.noreply.github.com> Co-authored-by: Perch2005 <78492631+Perch2005@users.noreply.github.com> Co-authored-by: Felipe Adachi <felipeadachi@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Throw exceptions if validation errors are detected when decoding model metrics protobuf messages.
Previously these were generating log messages that were easily overlooked in batch processing situations.
General Checklist
if it was a bug, test must cover it.