Throw exception if model metrics are incomplete #456
Closed
This pull request is big! We’re only showing the most recent 250 commits.
Commits on May 12, 2021
-
-
-
-
-
-
-
-
-
-
-
[FIX] Update property panel with non-discrete histogram data and some…
… minor style updates
-
-
-
-
-
-
Commits on May 18, 2021
-
Add profile viewer method (#229)
* move viewer to within whylogs * add profile viewer * 📚 add documentation for viewer * edits on profile viewer documentation * remove dividers * typos,and loom gif * use thumbnail gif from loom * Update README.md Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> * move metric collection list * Remove dots * Updating whylogs demo gif * Update README.md * Update README.md * Update README.md * bump version 0.4.8 Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> Co-authored-by: Sam Gracie <4944259+samgracie@users.noreply.github.com>
3 people committedMay 18, 2021
Commits on May 25, 2021
Commits on May 29, 2021
Commits on Jun 18, 2021
Commits on Jun 23, 2021
Commits on Jun 25, 2021
-
String features such as lenght, word count, character pos tracking (#236
-
serialize delimited format by default (#243)
Co-authored-by: Chris Warth <csw@whylabs.ai>
-
Commits on Jun 28, 2021
-
Co-authored-by: Andy Dang🤖 <andy@whylabs.ai>
Commits on Jul 8, 2021
-
-
-
-
fix imports add missing type removed unused variable format fix
-
-
-
-
-
Refactor datasetprofile's flatten profile (#250)
* add missing type removed unused variable format fix * refactor flatten summary functions * refactor flatten summary from datasetprofile * add more unit tests. * remove type conversion for strings * build with poetry for release * revert commit for type change * revert string type commit
Commits on Jul 9, 2021
Commits on Jul 10, 2021
-
* pre_commit make command * Autosegmentation feature * Update formatting with black * Autosegmentation reading from file * change print to log * style fixes * fix autosegmentation tests * remove unused import * auto seg notebook example * basic unit tests for segmentation function * add metadata config to unit testdata * fix metadata config reading and add s3 unit test for metadata * add empty test case * style fix Co-authored-by: Bernease Herman <bernease@gmail.com>
Commits on Jul 12, 2021
Commits on Jul 13, 2021
Commits on Jul 19, 2021
-
-
index on transport_parameters: 8a257f9 datasetprofiles reads and writ…
…es with smart_open
-
Check additional attribute in Logger (#260)
* check additional attribute to logger * additiona check on Logger instantiation * speed up ci slightly by testing faster os first
Commits on Jul 20, 2021
Commits on Jul 26, 2021
Commits on Jul 29, 2021
-
-
-
testing unicode unicode tracking add position test
-
-
-
-
-
Commits on Jul 30, 2021
-
remove default metadata writer
add None default value set default values to None instead of bool
Commits on Aug 5, 2021
Commits on Aug 13, 2021
Commits on Aug 14, 2021
Commits on Aug 19, 2021
-
[ADD] Handlebars config and basic sidebar structure
[ADD] Implement table data with hbs [FIX] Remove unused code [ADD] Two CDN and all-in-one versions of hbs templates [FIX] Typo in condition
-
compile and render page add register helpers check profile length
-
-
-
-
-
-
-
Commits on Aug 23, 2021
-
-
-
-
-
-
-
fix tests and remove duplicated subfolder
update test for faster performance fix notebook for tests style fixes compile protobuf files for test fix workflow remove example paths fix session variable make sure estimate segments doesn't write if there isnt a writer notebook fixes output json fixed
-
-
-
-
-
-
-
Commits on Aug 24, 2021
Commits on Aug 26, 2021
-
Sagemaker and Flask Example (#285)
* base for sagemaker container deployment * predictor code * style fixes * add flask app example * logroation fixes Closes #235 * increase time for demo * change notebook test name * Notebook for flask example * style fixes * ignore data folder * renamed sagemaker example * fix style * sagemaker model * log rotation * style and try join * updata canceling timers * wait for image logging to finish * rename flask example * skip flask notebook * update notebook * update flask app notes update
-
-
-
-
Commits on Aug 27, 2021
-
Stop JSONifying data. Bump schema minor version to 2
* Remove FrequentNumberSummary. This is superseded by frequent items data * Instead of JSONifying data, we just take a crude `str` approach. This ensures that we have consistent data in various environments. The previous behavior: the logic adds double quotes to string values.
Commits on Sep 1, 2021
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Sep 3, 2021
-
Fix model metric model_type switching and update tests (#302)
* Fix model metric model_type switching and update tests * Fix notebook example frequent strings reference
Commits on Sep 10, 2021
-
-
-
-
-
check if transport parameters is None remove verbose print remove minio mock style fixes fix test
-
-
Commits on Sep 14, 2021
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Sep 22, 2021
-
[IMPROVEMENT] Simply dependencies (#313)
* Move mlflow to test dependencies * Remove excel libraries (Pandas will detect and complain about it) Co-authored-by: Andy Dang🤖 <andy@whylabs.ai>
Commits on Sep 24, 2021
-
[IMPROVEMENT] Switch image profiling to using ImageStats (#315)
* [IMPROVEMENT] Switch image profiling to using ImageStats over per pixel metrics for better perf * Turn off the previous Hue Staturation Brightness on per pixel metrics * Switch to using ImageStat.Stats on image converted to HSV to log mean and stddev of Hue, Saturation, Brightness over the image * Updated tests and notebook so they still work * Added in a few more files of flowers so we can log more than one image to get some distributions in example notebook * Add logging for skipped exif keys * Remove wrapper class from transforms for Hue Saturation Brightness statistics
Commits on Sep 28, 2021
-
[MIGRATION] Move Java code base to whylogs (#318)
* Set up multipackage commit * Add proto module * Update submodule git * Remove whylogs java * Move file path to merge into a single repo * Remove publishing configuration in the core project * Don't wait for keystroke to exit * Use the DatasetProperties message in the DatasetSummary object * Minor fix for Java code * Fix java imports * Set up GitLab CI flow. Disable private Maven publishing for now. * Remove internal use package * Remove unused code * Set up Gradle build with submodule * Ignore .idea folder * [refactor] Simplified package name to com.whylogs * Update submodule with the latest protobof * Add PR workflow * Test PR CI * Run gradle when have pull request against mainline * [refactor] Rename Spark package * Add CLI module * Update CLI entry point * Remove the use of executor * Build Javadoc for the core project * Add LICENSE file * Preparing for publishing to Maven * Ignore Gradle.properties * [docs] Update whylogs-core README.md * Publish WhyLogs-Spark to Maven * Update documentation * Installation for WhyLogs-Spark * Add instruction for Apache Spark integration * [docs] Update to using new classpath * Update README.md Unifying the java readme with the python readme for consistency of experience * Update README.md One more time, updating the readme to be consistent across python & java * Update Schema to 1.1: use maps for tagging. * [docs] Update doc with new tag API * [docs] Update with OpenJDK shield * [config] Add bumpversion config * Bump version: 0.0.2b1 → 0.0.2b2 * [docs] Update to new version * Edited wording Edited to align with wording in the Python readme * Moved Python crosslink to start of doc * Added glossary * Fixed Profiler * Add performance benchmark. Move Glossary to the top * Add a section about statistical profile * Move Performance to a sub-section of the intro * Minor Readme fix * Update the Spark example for better documentation * Increase string tracker to track 128 items instead of 32 * Bump version with increase of default item size * Run linting * Fix README.md header styling * * Update Spark example * Handle null value when creating tags from row * Handle null tags and null columns in DatasetProfile's constructor * Clarify Spark version support * Update description for packages * Update description for packages * Bump version: 0.0.2b2 → 0.1.0b0 * Bump version: 0.1.0b0 → 0.1.0 * Add Link to repo. Update version in README.md * Add Link to repo. Update version in README.md * Publish JavaDoc * Align with whylogs-python * Update naming + description in publishing script * Remove StringTracker. The frequent item tracking logic is moved to the top level in ColumnProfile * Add Frequent Items and Cardinality Tracker to Column Profile * Bump version: 0.1.0 → 0.1.1b0 * Bump version: 0.1.1b0 → 0.1.1 * Fix type in our example Was declaring a map as a list * Upgrade to Spark 3.0 and Scala 2.12 * Fixed dataset profile test. Our precision only goes up to milliseconds, so we should only verify Instant's milliseconds value * Output a Array[Byte] instead of a wrapper class This will enable the Python code to access the APIs without deal with schema creation * Add logic to build a Spark fat jar This fat jar makes it easy to test and experiment with whylogs and Spark * Remove unused code * Add a Java-friendly API for groupBy * Add Python bridge for accessing a whylog profiling session * Add helper method to write data to parquet format * Wrap the JDF in Dataframe before returning * Align whylogs-java with whylogs-python * Add frequent number to the tracker * Using string tracker to track frequent number * Add theta sketch to number tracker * Update bump2version * Bump version: 0.1.1 → 0.1.2-b0 * Remove shadowjar in spark * Add classification metrics for binary and multiclass classification models * Adding support for classification metrics in whylogs Spark * Use a streaming algorithm with matrix expansion * Track prediction scores in a NxN matrix * Track prediction scores in a matrix of sketches * Track boolean predictions * Add Python binding for Scala * Update README.md Fixed minor spelling error * Add classification metrics for binary and multiclass classification models * Adding support for classification metrics in whylogs Spark * Use a streaming algorithm with matrix expansion * Track prediction scores in a NxN matrix * Track prediction scores in a matrix of sketches * Track boolean predictions * Add Python binding for Scala * Do not exclude classifcation metrics from the dataset profile * Update to latest protobuf * Update to the latest schema with ModelProfile under `DatasetProfile` * Update the minor version * Run aggregation and publish to WhyLabs * Ensure that we copy the model profile over when performing Java ser/der * Fix handling new labels * Fixed a bug where we're dropping model metrics during Spark reducing Add end to end test for model metrics * Add tests for confusion matrix * Bump version: 0.1.2-b0 → 0.1.2-b1 * Fix deserialization bug where we create an empty ModelProfile when deserialize from protobuf * Bump version: 0.1.2-b1 → 0.1.2-b2 * Fixed bug where data is doubled indexed when profile numeric text * Bump version: 0.1.2-b2 → 0.1.2-b3 * Fixed cardinality tracker * Return 0 when count is 1 * Bump version: 0.1.2-b3 → 0.1.2-b4 * Run spotlessApply target * Fixed ScoreMatrixTest * Adding RegressionMetrics * Add regression metric support * Fixed serialization bug of counter tracker. Add serialization test * Make the code Scala 2.11 friendly (for Spark 2.x) * Add test for null count * Track NaN value as null value * Parameterize scalaVersion and sparkVersion * Enable specifying custom datetime when calling `session.log` * Use String.format to avoid relying on Guava 20 API * Use Guava 19 to make whylogs compatible with most systems * Bump version: 0.1.2-b4 → 0.1.2-b5 * Add findbugs dependency for javax.annotation.Nullable * Run spotlessApply * Stop using Guava Streams API * user can specify null check values * spotlessApply format changes * keep nullstr overrides per-instance - avoids test of nullstr functionality from affecting other tests * require empty null str set - don't allow nullstrs to be null. - convert from ImmutableList top Sets * test for empty list & fix ColumnProfile.builder usage * switch to use ImmutableSet * set nullstr in fromProtobuf * set nullstrs in builders * user can specify null check values * Bump version: 0.1.2-b0 → 0.1.2-b1 * Updated with latest published version * Bump version: 0.1.2-b5 → 0.1.2-b6 * Enable building for multiple spark version * Bump version: 0.1.2-b6 → 0.1.2-b7 * Using public WhyLabs client * Enable overriding WhyLabs endpoint for testing * Adding missing okhttp3 * Update .log API to allow customizing endpoints * Calculate quantiles when building NumberSummary * spotless format changes * surround for-loop body in {} * Druid aggregators have to pre-allocate bytearrays for intermediate results so we need a sane limit on our per-string char count in order to math out the largest intermediate aggregation bucket size * fmt * Update README.md Update Java doc link * Fix mismatch null count and schema count Backwards compatibility: summary still has the null count under "counters" For existing profiles, we can workaround by adding the two values together. For new profiles after this PR, the null_count will always come from SchemaTracker and not Counters Also, turn off SCHEMA detection by default Reference: #234 * Use the new upload url based api for logging Our original upload api has the client uploading directly to our rest service. Now we return upload urls that the client should use to upload directly, rather than proxying through our servers. * Fix file path in upload api Was using the file name instead of the file path. * Add generic retry utility and make profile uploads retry A lot of resources online point towards leaning on akka to handle things like this but it seems overkill as a dependnecy for this use case. The one thing I need to confirm before pushing is the relationship between futures and threads in scala. In Kotlin, I would just be using a coroutine specific delay that I know won't deadlock. Here, if its true that every future runs on its own thread then this should be fine. If that isn't true then it can be risky. * add unit test of MetricsSummary JSON encoding * Update API to distinguish between Classification vs Regression model (#29) * Update API to distinguish between Classification vs Regression model * Enable tagging with whylogs.tag prefix Co-authored-by: Andy Dang🤖 <andy@whylabs.ai> * add Getters to all model metrics fields. - make ModelMetrics.fromProtobuf match python functionality. * don't add identical labels to confusion matrix * deduplicate in the more obvious way * bump to version 0.1.3-b2 (#32) Co-authored-by: Chris Warth <csw@whylabs.ai> * update StringTracker with new fields from python library * PR feedback - add comments, simplify some for-loops * fix NPE in StringTracker.merge * fix NPE in StringTracker.toProtoBuf * make StringTracker NonNull - simplify backward compatibility testing in fromProtobuf * Fixed typo for withRegressionModel method (#35) * Fix small typo * Bump version: 0.1.3-b2 → 0.1.3-b3 Co-authored-by: Andy Dang🤖 <andy@whylabs.ai> * PySpark autosegmentation and tests (#36) * [WIP] Adding dependencies to the python folder This is a POC to get unit testing enabled in a sane way for the python code that we have. Running `make install test` should get the sample test running. After, we can make sure to run that command during the build as well. * Fixed typo for withRegressionModel method (#35) * Add PySpark autosegmentation and tests Co-authored-by: Anthony Naddeo <anthony@whylabs.ai> Co-authored-by: Leandro G. Almeida <leandro.g.almeida@gmail.com> * Upgrade to whylabs-api-client 0.1.7 to use the latest API signature * Bump version: 0.1.3-b3 → 0.1.3 * protect against exception when scorematrix has no labels * protect against divide-by-zero under more specific conditions * Avoid NPEs when merging profiles with null model metrics * make stringtracker merge robust for missing data (#41) Co-authored-by: Chris Warth <csw@whylabs.ai> * Fixed profile merge bugs inside regression metrics and string tracker * fmt * Update core/src/main/java/com/whylogs/core/DatasetProfile.java Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> * typo * NPE on copy * Profile merge regressions (#42) * Fixed profile merge bugs inside regression metrics and string tracker * fmt * Update core/src/main/java/com/whylogs/core/DatasetProfile.java Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> * typo * NPE on copy Co-authored-by: drew dahlke <drew@drews-mbp.attlocal.net> Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> * Found another issue when merging legacy profiles from very early days * Fixed some serde issues with stringtracker * fmt * Profile merge bug on very old profiles (#43) * Fixed profile merge bugs inside regression metrics and string tracker * fmt * Update core/src/main/java/com/whylogs/core/DatasetProfile.java Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> * typo * NPE on copy * Found another issue when merging legacy profiles from very early days * Fixed some serde issues with stringtracker * fmt Co-authored-by: drew dahlke <drew@drews-mbp.attlocal.net> Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> * Older versions of profiles could have model metrics populated without having the metric model type populated because validation didn't get added until later. Ensure these profiles can still be merged * I don't think this can be null, but just to be safe * Histogram summary (#46) * Bump version: 0.1.3 → 0.1.4-b6 * update summary histogram calculation * histogram splitpoints unit test Co-authored-by: Chris Warth <csw@whylabs.ai> * Major perf improvement: support very wide Spark dataframe (#47) * Align metrics name. Remove duplicate classes * More information about API exception * Remove frequentItems from NumberTracker This information is capture in the top level frequent item tracker within a column * Optimize for handling very wide Spark dataframes This allows users to avoid Spark blowing up * Update Schema minor version * Adding more tests around model metrics and features * Bump version: 0.1.4-b6 → 0.1.5-b0 * Better null handling when parsing Protobuf messages * Add more annotation around nullable vs nonull * Minor perf optimization * Add support for snapshot repo * Add a simple roundtrip test * fix styling Co-authored-by: Andy Dang🤖 <andy@whylabs.ai> * Update proto build logic and align schemas * Remove proto submodule * Formatting fix * Adding gradle github action * Re-instate NLP Message * More style fix * Temporarily ignore pyspark test Co-authored-by: Andy Dang🤖 <andy@whylabs.ai> Co-authored-by: Alessya Labzhinova <alessya.l@gmail.com> Co-authored-by: DialMforMara <32914820+DialMforMara@users.noreply.github.com> Co-authored-by: Anthony Naddeo <anthony@whylabs.ai> Co-authored-by: Chris Warth <csw@whylabs.ai> Co-authored-by: Chris Warth <cswarth@gmail.com> Co-authored-by: drew dahlke <drew@drews-mbp.attlocal.net> Co-authored-by: Bernease Herman <bernease@gmail.com> Co-authored-by: Leandro G. Almeida <leandro.g.almeida@gmail.com> Co-authored-by: naddeoa <anthony.naddeo@gmail.com> Co-authored-by: Drew Dahlke <drew@totvs.com>12 people committedSep 28, 2021
Commits on Sep 30, 2021
Commits on Oct 1, 2021
-
Add regex pattern matching to ValueConstraint (#317)
Co-authored-by: Felipe Adachi <e-felipea@weg.net> Co-authored-by: Andy Dang <26821974+andyndang@users.noreply.github.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
4 people committedOct 1, 2021 -
Merge new examples to mainline from Dev/loka/examples (#316)
* flask and whylabs example first iteration * added script that trains a sklearn model * added script that trains a sklearn model * ✨ Added first version of Flask example * update sagemaker example * swagger documentation added * TODO added * flask example updated * fix bug with .gitignore and requirements.txt * fix bug with .gitignore and requirements.txt * feature vector schema changed passing a field per feature * swagger feature vector model modified * ✨ Now Session never ends :D * ✨ sagemaker sklearn example working * 📝 update documentation * ✨ clean up script for sagemaker endpoint added * ✨ Added atexit routine and scheduled update df action * 🐛 fix bugs with sagemaker whylabs example * ✨ jupyter notebook for sagemaker example added * 📝 flask example documentation updated * 📝 added more docs on flask example * minor updates * 🎉 First commit testing mlflow example * 📝 Readme enhanced with session management * 📝 documentation fix * 📝 flask whylabs docs finished * fix some bugs * update docs on sagemaker example * sagemaker fix * replace downloading in flask sklearn example * update sagemaker docs * rename folders * sagemaker example improved * ✨ Modified whylabs yaml * ✨ Initial version MLFLow writter * ✨ Getting Rotation time from .env * added first example notebook for mlflow whylabs integration * clean code * ✨ Fixed Session management inside patche * 🔥 Removed threading behavior in MLFlowWriter Class * readme first update * 🐛 Revert changes in patcher about session * mlflow exampled updated * readme sagemaker fix * sagemaker constant requests added * make precommit run * unnecesary mlflow tests removed * Sagemaker example not initializing for every request * most of the tests are working * all tests are working * fix format * Fix skip_notebooks Co-authored-by: nroldanf <nicolas.roldan@mail.escuelaing.edu.co> Co-authored-by: jjpulidos <jjpulidos@unal.edu.co>
3 people committedOct 1, 2021
Commits on Oct 4, 2021
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
-
Update API token when updated in the env variable and recreate client…
…, more verbose output on upload (#327)
-
Add warning on larger dataframe logging, add perf test and fixture fo…
…r conditional tests w/ --load (#326) * Add test fixture for conditionally running test if --load_test is passed, and add a test to profile log_dataframe * Add warning message if logging a dataframe larger than 200k elements
Commits on Oct 7, 2021
-
Changed MATCH and NOMATCH to use re.compile instead of re.match. Chan…
…ged name of constraint in test_constraints.py to not_contains_date. (#325) Co-authored-by: Felipe Adachi <e-felipea@weg.net> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedOct 7, 2021 -
Commits on Oct 12, 2021
-
[API] Simplify WhyLabs writer + deprecate anonymous writer (#333)
* Simplify WhyLabs writer * Remove the anonymous session upload logic * Deprecate start_whylabs_session and data_collection_consent * Relax WriterConfig validation (in case users still have the old config) Co-authored-by: Andy Dang🤖 <andy@whylabs.ai>
-
Commits on Oct 14, 2021
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Oct 19, 2021
-
-
-
Merge pull request #342 from whylabs/dev/jamie/session_default
Make project name and pipeline name optional parameters for session
Commits on Oct 25, 2021
-
Update version to 0.6.9 (#343)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Oct 27, 2021
-
Enable pickling via protobuf (#344)
* Enable pickling via protobuf We were never able to pickle our dataset profile objects. Our integration work with Ray made this a pressing issue since Ray depends heavily on calling pickle on objects. This change will enable pickle by serializing/deserializing to/from protobuf. In doing this, I stumbled upon other actual issues, like we apparently don't deserialize from protobuf correctly in certain cases, like when we have no set dataset_timestamp. It ends up as a 1970 date. I have two tests that are omitted that I'm going to work on next to fix that issue. * Fix some date serialization bugs Found a few issues around protobuf serde with dates that would cause the deserialized profiles to have 1970 dataset timestamps or have otherwise subtlyl different datetime state. Co-authored-by: Anthony Naddeo <anthony@whylabs.ai>
Commits on Oct 28, 2021
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Nov 4, 2021
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Nov 9, 2021
Commits on Nov 12, 2021
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Nov 16, 2021
-
Minor docstream, filename, error message fixes (#356)
* Minor fixes to docstrings, error message, filename... * `feature_name` is listed twice in a couple places * `s/Unambiguous/Ambiguous/` in exception -- check this is correct * `mv test_diplay.py test_display.py` * another feature_name glitch
-
Put in some guardrails against overly large confusion matrices (#355)
* Put in some guardrails against overly large confusion matrices
-
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Nov 18, 2021
-
Black seems to be pulling in versions of typed-ast that break unless constrained to typed-ast >= 1.5
Commits on Nov 23, 2021
Commits on Nov 30, 2021
-
Fix exception related to logging array like column values (#365)
* Fix exception related to logging array like column values
-
Use log_async in WhylabsWriter (#359)
* Use log_async in whylabs writier Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Commits on Dec 10, 2021
-
Spelling issue, formatting issue, and added a comment to build instructions. `make install` is usually reserved for installing something into your system but we use it for installing our required build dependencies, which I regret, so I added a comment to clarify since it looked like we put our build instructions backwards.
Commits on Dec 13, 2021
-
Fix strings as arrays pd.isnull bug (#372)
* Fix bug where strings parsed as array was passed to pd.isnull
Commits on Dec 14, 2021
-
-
Dev/loka/btwn constraints (#374)
* Added BTWN operation in Op enum, new Op.BTWN lambda summary function, edit SummaryConstraint class and SummaryConstraintMsg accordingly, unit tests for BTWN * Created stddevBetweenConstraint, meanBetweenConstraint, minBetweenConstraint, minGreaterThanEqualConstraint, maxBetweenConstraint and maxLessThanEqualConstraint. Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: MilenaTrajanoska <trajanoska28@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
4 people committedDec 14, 2021 -
GitHub action to publish Java/Scala (#370)
* GitHub action to publish Java/Scala
Commits on Dec 15, 2021
Commits on Dec 16, 2021
-
Update version to release 0.6.17 (#377)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
-
ValueConstraint: Assert column values in set (#375)
* Added BTWN operation in Op enum, new Op.BTWN lambda summary function, edit SummaryConstraint class and SummaryConstraintMsg accordingly, unit tests for BTWN * Created stddevBetweenConstraint, meanBetweenConstraint, minBetweenConstraint, minGreaterThanEqualConstraint, maxBetweenConstraint and maxLessThanEqualConstraint. * Created IN operator, added value_set to ValueConstraintMsg, created columnValuesInSetConstraint. * Changed columnValuesInSetConstraint type of parameter value_set to Set[Any], simplified condition for checking constraints with IN operator, added assertions and logger in test cases for constraints * Changed Op.IN to Op.IN_SET Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: MilenaTrajanoska <trajanoska28@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
4 people committedDec 16, 2021
Commits on Dec 17, 2021
Commits on Dec 21, 2021
-
Value Constraint: Contains specific pattern constraint (#378)
* Created emailConstraint initial implementation * Created containsCreditCardConstraint initial implementation * Implemented containsSSNConstraint and containsURLConstraint. * Add update_typed method in ValueConstraints * Add raw_value_constraints and coerced_type_constraints in ValueConstraints Co-authored-by: pecop2 <petar@loka.com>
-
Commits on Dec 22, 2021
-
Update version to release 0.6.18 (#395)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
-
SummaryConstraint: Assert distinct values in set constraint (#379)
* Assert distinct values in set constraint. New Ops in proto for SET: IN, CONTAINS, EQ. New lambda functions for the operations, using string_tracker and number_tracker theta sketches for distinct column values set operations * Specific functions for each SET constraint * Edit constraints.proto to change the Enum for the Op.CONTAINS to Op.CONTAIN. Add serialization test for SET summary constraints Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedDec 22, 2021
Commits on Dec 28, 2021
-
Remove git submodules in Java workflows (#396)
* Remove git submodules in Java workflows * Bump log4j to 2.17.0 in Java package * Update readme * add jupyterlab to docker dev environment Co-authored-by: Richard Rogers <rrogers@opendius.net>
Commits on Dec 30, 2021
-
Summary Constraint: quantileBetweenConstraint (#381)
* Implemented quantileBetweenConstraint * Added quantile_value argument in SummaryConstraint * Add summary converter single_quantile_from_sketch utility function, refactor summary constraints update object * Change proto number of quantile_value, update logic in SummaryConstraint update method * Simplified dict update in _create_update_summary_dictionary function Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
-
Update version to 0.6.19 (#399)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Commits on Jan 4, 2022
-
ValueConstraint: Datetime/json constraints (#387)
* Assert dateutil parseable. Implemented with adding an APPLY_FUNC operation and passing a custom function to the ValueConstraint for later evaluation * Dateutil parseable, APPLY_FUNC * Assert string length constraint. Length equals or is between two values. * Fix from_protobuf when using APPLY_FUNC op, serialization tests * Create specific dateUtilParseableConstraint function * JSON parseable constraint initial * Assert dateutil parseable, json parseable, match json schema * Tests for strftime format and change ValueConstraints update of pre and post typed data * Fix tests and bad merge * Proto add APPLY_FUNC in enum * Change OP from CONTAINS_SET to CONTAIN_SET. Co-authored-by: pecop2 <petar@loka.com>
-
SummaryConstraint: columnUniqueValueCountBetweenConstraint, columnUni…
…queValueProportionBetweenConstraint (#391) * Created emailConstraint initial implementation * Created containsCreditCardConstraint initial implementation * Implemented containsSSNConstraint and containsURLConstraint. * Changed constraints to be evaluated before typed data conversion * Case insensitive containsEmailConstraint * Add update_typed method in ValueConstraints * Implemented columnUniqueValueCountBetweenConstraint and columnUniqueValueProportionBetweenConstraint Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedJan 4, 2022
Commits on Jan 5, 2022
Commits on Jan 7, 2022
-
Update version to 0.6.20 (#407)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Jan 8, 2022
-
-
* Java package publishing via CI * Bump log4j to 2.17.0 in Java package Co-authored-by: Richard Rogers <rrogers@opendius.net> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedJan 8, 2022
Commits on Jan 13, 2022
-
Implemented columnMostCommonValueInSetConstraint (#390)
* Implemented columnMostCommonValueInSetConstraint * Refactored code to use _create_update_summary_dictionary function for updating the summary constraints * Refactor name, merge, to_protobuf and from_protobuf in ValueConstraint and SummaryConstraint Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Commits on Jan 14, 2022
-
Implemented columnValuesNotNullConstraint (#392)
* Implemented columnValuesNotNullConstraint * Refactored code to use _create_update_summary_dictionary function for updating the summary constraints Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Commits on Jan 17, 2022
-
SummaryConstraint: columnValuesTypeEqualsConstraint and columnValuesT…
…ypeInSetConstraint (#393) * Implemented columnValuesTypeEqualsConstraint and columnValuesTypeInSetConstraint * Merge and refactor to_protobuf, from_protobuf, merge, name methods in ValueConstraint and SummaryConstraint, refactor tests * Cast unique count estimate to integer, since the constraint accepts only integer values for comparison.
Commits on Jan 18, 2022
-
Implemented approximateEntropyBetweenConstraint (#394)
* Implemented approximateEntropyBetweenConstraint * Add documentation for parameter histogram in summaryconverters function entropy_from_column_summary
Commits on Jan 20, 2022
-
SummaryConstraints: Distributional Measures Constraints (#397)
* Implemented parametrizedKSTestPValueGreaterThanConstraint * Add documentation for parameter reference_distribution in SummaryConstraint * Implemented columnKLDivergenceLessThanConstraint for continuous and discrete distributions * Implemented columnChiSquaredTestPValueGreaterThanConstraint
Commits on Jan 22, 2022
Commits on Jan 25, 2022
-
-
Add scipy for recent constraints work (#425)
* Add scipy for recent constraints work, dataclasses for python < 3.7
-
Integrate wheel smoke test into standard CI workflow. (#422)
* Integrate wheel smoke test into standard CI workflow. Co-authored-by: Richard Rogers <rrogers@opendius.net> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedJan 25, 2022 -
SummaryConstraint: Table shape constraints (#398)
* Table shape constraints as SummaryConstraint, changes in datasetprofile needed for implementation, modify DatasetConstraints accordingly, tests for table shape constraints and DatasetConstraints serialization * Add the table shape constraints report to the DatasetConstraints report. Modify the DatasetConstrains serialization test. * Change OP CONTAINS_SET to CONTAIN_SET. Add function to simplify SummaryConstraint.from_protobuf. * Change the approach for the table shape number of rows constraint. Tests for sequential logging. Parameterize set length. Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedJan 25, 2022
Commits on Jan 26, 2022
-
MultiColumn Value Constraints (#413)
* Multicolumn value constraints, binded columns at constraint creation time - initial implementation * Changed to_protobuf method in MultiColumnValueConstraint to return MultiColumnValueConstraintMsg * columnPairValuesInSetConstraint initial implementation * Changes in proto MultiColumnValueConstraintMsg, to_protobuf and from_protobuf in MultiColumnValueConstraint * sumOfRowValuesOfMultipleColumnsEqualsConstraint initial implementation * Added internal_op attribute in MultiColumnValueConstraint * Logical opreations constraint. New class for MultiColumnValueConstraint and MultiColumnValueConstraints. Change the way of tracking the dataframe at DatasetProfile. Constraint for column_values_A_greater_than_B and simple test for it. * Tests for the multi column value constraints, changes in from_protobuf and to_protobuf of MCVC, change OP CONTAINS_SET to CONTAIN_SET. Constraint functions for all the logical operations between two columns. * Implemented columnPairValuesInSetConstraint * Implemented columnValuesUniqueWithinRow Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedJan 26, 2022 -
Update version to 0.6.24 (#426)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
-
Constraints example notebook (#417)
* Example notebook for the newly created constraints * Added Chi squared constraint example supplying a map of items and frequencies as counts * Change values in some examples with distributional measures * Generate constraints possible constraints * Add tests for generate_constraints, example for generate_constraints in Constraints_Suite.ipynb * Added new example for table shape constraints and sequentially logging data more than once Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: MilenaTrajanoska <trajanoska28@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
4 people committedJan 26, 2022
Commits on Jan 27, 2022
-
Change contraints'total row number computation (#427)
* Change total row number computation * Edge case not to throw a ValueError when no columns logged Co-authored-by: pecop2 <petar@loka.com>
Commits on Feb 2, 2022
Commits on Feb 8, 2022
-
Update version to release 0.6.25 (#430)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
-
Display human readable name for constraints (#431)
* Display user supplied name or default human-readable name for the constraints * Add to_protobuf method in MultiColumnValueConstraints Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
-
Update version to release 0.6.26
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Feb 9, 2022
-
Missing values proportion between constraint (#436)
* Implemented missingValuesProportionBetweenConstraint
Commits on Feb 10, 2022
-
Constraints Documentation (#435)
* Add documentation for name in constraints, Document summary converters Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Commits on Feb 15, 2022
-
DatasetConstraints serialization and chi square calculation fixes (#437)
* Change MultiColumValueConstraints serialization * Fix Chi-Squared statistic calculation * Edit getattr to hasattr * Fix Chi-Squared statistic calculation * Edit test_dataset_constraints_serialization * Fix ZeroDivisionError in kl divergence computation * Pass a deep copy of the column values in the MultiColumnValueConstraints update method to not accidentally change them during update * Add almost zero probability of event variable in _compute_kl_divergence_continuous_distributions Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: MilenaTrajanoska <trajanoska28@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
4 people committedFeb 15, 2022 -
Add Java package smoketest (#438)
* Add Java package smoketest Co-authored-by: Richard Rogers <rrogers@opendius.net> Co-authored-by: Andy Dang🤖 <andy@whylabs.ai> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
4 people committedFeb 15, 2022 -
Distributional Measures Constraints Example Notebook (#440)
* Constraints distributional measures example use-case * Add comment in example notebook Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Commits on Feb 17, 2022
-
Update version to release 0.6.27 (#443)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Feb 18, 2022
Commits on Feb 21, 2022
-
Fix KS test p-value computation (#444)
* Fix KS test p-value computation, add condition in entropy * Change number of samples calculation Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
-
Constraints example notebook typos fix (#449)
* Fix typos and add example for the missing values proportion constraint in Constraints_Suite Co-authored-by: pecop2 <petar@loka.com> Co-authored-by: MilenaTrajanoska <trajanoska28@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
4 people committedFeb 21, 2022
Commits on Feb 22, 2022
Commits on Feb 23, 2022
-
Update version to release 0.6.28 (#453)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Commits on Feb 25, 2022
Commits on Feb 28, 2022
-
Add feast integration demo (#454)
* Add feast integration demo * - Removed parquet files from the repo and moved to s3 * add .gitkeep to push empty data folder
Commits on Mar 3, 2022
-
Cleaned up typos, added Contributors subsection, centered badges, added a section describing what can be done with profiles
-
Update version to 0.6.29 (#463)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
Commits on Mar 8, 2022
-
-
Fix missing dataTimestamp post aggregation (#465)
* Fix missing dataTimestamp post aggregation * Add a scala test to cover deserializing the dataTimestamp from the DatasetProfile
Commits on Mar 9, 2022
Commits on Mar 14, 2022
-
Update version to 0.6.30 (#467)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
-
Performance Metrics examples (#469)
* add performance metrics - classification and regression - notebooks
Commits on Mar 15, 2022
-
Matrix build for Spark and Scala versions (#447)
* Matrix build for Spark and Scala versions * add spark 3.1.1 * publish matrix builds * include Spark 3.2 & Scala 2.13 * need os in included matrix entry * Scala 2.13 will need code changes * match publish matrix to build matrix * spark/scala default versions Co-authored-by: Richard Rogers <rrogers@opendius.net> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com>
3 people committedMar 15, 2022 -
Dev/loka/jupyter notebook visualization (#433)
* visualizer new design, add reference profile in profile viewer * add json reference profile button, add UI changes * change view detail button position, change referenceProfilePanel chart view, add icons * add property panel for reference profile, charts for json profile without reference * add the notification sign of filter button * use addFileName() function when the file is load * add "Filter by type" design changes, add "Quick search" design changes. * add "Quick search" input. * count of types in "filter by type" dropdown * add double histogram chart in reference profile panel. * "Sign up for a free WhyLabs account" button disappears when screen with less than 1000px. * add reference profile charts, add profile name and time stamp * add diff from ref when reference profile consists. * Shows alert when the key of profile data is not matched to reference profile data. * histograms visualization in jupyter notebook * add backend for jupyter notebook vizualisation, add jupyter notebook to display profile data in the notebook * add double histogram visualization * responsive distribution chart * add search logic * add folder for saved htmls, add all methods to jupyter notebook cells, change backend code stucture * add frequent items in reference property panel * add feature summary statistics backend, add single profile summary statistics backend * add Full Profile Summary Statistics visualization * add Feature Summary Statistics visualisation * add ability to add multiple features in double histogram. * show full number at hover on statistics numbers * add documentation in notebook, naming fixes * add calculations in backend for variance and coefficient of variation * add constraints report backend, add constraints report in notebook * constraints report with real values * Categorical drift report: Differenced Bar chart jupyter visualization * Add definition (ranges) for the severe, moderate, low drift categories. * change import in tests, add categorical profile generator in tests * change css and js paths for browser visualizer * Histogram Bin Width and Range (#468) - Changed upper value of domain's xscale to match the max between both distribution's -> max of bin value + bin_width - apply Xscale to each of distribution's [1] and [0] before calculating bar width, instead of scaling the difference ([1]-[0]) Co-authored-by: Perch <perchgithub@gmail.com> Co-authored-by: ssxcho <xchton@gmail.com> Co-authored-by: Jamie Broomall <88007022+jamie256@users.noreply.github.com> Co-authored-by: Khachatur Tonoyan <31924981+ssxcho@users.noreply.github.com> Co-authored-by: Perch2005 <78492631+Perch2005@users.noreply.github.com> Co-authored-by: Felipe Adachi <felipeadachi@gmail.com>
7 people committedMar 15, 2022 -
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.