title |
---|
Changelog |
- [FEATURE] Split data assets using sql datetime columns (#4871)
- [FEATURE] Plot metrics with
DataAssistantResult.plot()
(#4873) - [FEATURE] RuleBasedProfiler/DataAssistant/MetricMultiBatchParameterBuilder: Enable Returning Metric Computation Results with batch_id Attribution (#4862)
- [FEATURE] Enable variables to be specified at both Profiler and its constituent individual Rule levels (#4912)
- [FEATURE] Enable self-initializing
ExpectColumnUniqueValueCountToBeBetween
(#4902) - [FEATURE] Improve diagnostic testing process (#4816)
- [FEATURE] Add Azure CI/CD action to aid with style guide enforcement (type hints) (#4878)
- [FEATURE] Add Azure CI/CD action to aid with style guide enforcement (docstrings) (#4617)
- [FEATURE] Use formal interfaces to clean up DataAssistant and DataAssistantResult modules/classes (#4901)
- [BUGFIX] fix validation issue for column domain type and implement expect_column_unique_value_count_to_be_between for VolumeDataAssistant (#4914)
- [BUGFIX] Fix issue with not using the generated table name on read (#4905)
- [BUGFIX] Add deprecation comment to RuntimeDataConnector
- [BUGFIX] Ensure proper class_name within all RuleBasedProfilerConfig instantiations
- [BUGFIX] fix rounding directive handling (#4887)
- [BUGFIX]
great_expectations
import fails when SQL Alchemy is not installed (#4880) - [MAINTENANCE] Altair types cleanup (#4916)
- [MAINTENANCE] test: update test time (#4911)
- [MAINTENANCE] Add module docstring and simplify access to DatePart (#4910)
- [MAINTENANCE] Chip away at type hint violations around data context (#4897)
- [MAINTENANCE] Improve error message outputted to user in DocstringChecker action (#4895)
- [MAINTENANCE] Re-enable bigquery tests (#4903)
- [MAINTENANCE] Unit tests for sqlalchemy splitter methods, docs and other improvements (#4900)
- [MAINTENANCE] Move plot logic from
DataAssistant
intoDataAssistantResult
(#4896) - [MAINTENANCE] Add condition to primary pipeline to ensure
import_ge
stage doesn't cause misleading Slack notifications (#4898) - [MAINTENANCE] Refactor
RuleBasedProfilerConfig
(#4882) - [MAINTENANCE] Refactor DataAssistant Access to Parameter Computation Results and Plotting Utilities (#4893)
- [MAINTENANCE] Update
dgtest-overrides
list to include all test files not captured by primary strategy (#4891) - [MAINTENANCE] Add dgtest-overrides section to dependency_graph Azure pipeline
- [MAINTENANCE] Datasource and DataContext-level tests for RuntimeDataConnector changes (#4866)
- [MAINTENANCE] Temporarily disable bigquery tests. (#4888)
- [MAINTENANCE] Import GE after running
ge init
in packaging CI pipeline (#4885) - [MAINTENANCE] Add CI stage importing GE with only required dependencies installed (#4884)
- [MAINTENANCE]
DataAssistantResult.plot()
conditional formatting and tooltips (#4881) - [MAINTENANCE] split data context files (#4879)
- [MAINTENANCE] Add Tanner to CODEOWNERS for schemas.py (#4875)
- [MAINTENANCE] Use defined constants for ParameterNode accessor keys (#4872)
- [FEATURE] Additional Rule-Based Profiler Parameter/Variable Access Methods (#4814)
- [FEATURE] DataAssistant and VolumeDataAssistant classes (initial implementation -- to be enhanced as part of subsequent work) (#4844)
- [FEATURE] Add Support for Returning Parameters and Metrics as DataAssistantResult class (#4848)
- [FEATURE] DataAssistantResult Includes Underlying Profiler Execution Time (#4854)
- [FEATURE] Add batch_id for every resolved metric_value to ParameterBuilder.get_metrics() result object (#4860)
- [FEATURE]
RuntimeDataConnector
able to specifyAssets
(#4861) - [BUGFIX] Linting error from hackathon automerge (#4829)
- [BUGFIX] Cleanup contrib (#4838)
- [BUGFIX] Add
notebook
toGE_REQUIRED_DEPENDENCIES
(#4842) - [BUGFIX] ParameterContainer return value formatting bug fix (#4840)
- [BUGFIX] Ensure that Parameter Validation/Configuration Dependency Configurations are included in Serialization (#4843)
- [BUGFIX] Correctly handle SQLA unexpected count metric for empty tables (#4618) (thanks @douglascook)
- [BUGFIX] Temporarily adjust Deprecation Warning Count (#4869)
- [DOCS] How to validate data with an in memory checkpoint (#4820)
- [DOCS] Update all tutorial redirect fix (#4841)
- [DOCS] redirect/remove dead links in docs (#4846)
- [MAINTENANCE] Refactor Rule-Based Profiler instantiation in Validator to make it available as a public method (#4823)
- [MAINTENANCE] String Type is not needed as Return Type from DomainBuilder.domain_type() (#4827)
- [MAINTENANCE] Fix Typo in Checkpoint Readme (#4835) (thanks @andyjessen)
- [MAINTENANCE] Modify conditional expectations readme (#4616) (thanks @andyjessen)
- [MAINTENANCE] Fix links within datasource new notebook (#4833) (thanks @andyjessen)
- [MAINTENANCE] Adds missing dependency, which is breaking CLI workflows (#4839)
- [MAINTENANCE] Update testing and documentation for
oneshot
estimation method (#4852) - [MAINTENANCE] Refactor
Datasource
tests that work withRuntimeDataConnector
by backend. (#4853) - [MAINTENANCE] Update DataAssistant interfaces (#4857)
- [MAINTENANCE] Improve types returned by DataAssistant interface methods (#4859)
- [MAINTENANCE] Refactor
DataContext
tests that work with RuntimeDataConnector by backend (#4858) - [HACKATHON] Hackathon PRs in this release
- [BREAKING] EOL Python 3.6 (#4567)
- [FEATURE] Implement Multi-Column Domain Builder for Rule-Based Profiler (#4604)
- [FEATURE] Update RBP notebook to include example for Multi-Column Domain Builder (#4606)
- [FEATURE] Rule-Based Profiler: ColumnPairDomainBuilder (#4608)
- [FEATURE] More package contrib info (#4693)
- [FEATURE] Introducing RuleState class and RuleOutput class for Rule-Based Profiler in support of richer use cases (such as DataAssistant). (#4704)
- [FEATURE] Add support for returning fully-qualified parameters names/values from RuleOutput object (#4773)
- [BUGFIX] Pass random seed to bootstrap estimator (#4605)
- [BUGFIX] Adjust output of
regex
ParameterBuilder to match Expectation (#4594) - [BUGFIX] Rule-Based Profiler: Only primitive type based BatchRequest is allowed for Builder classes (#4614)
- [BUGFIX] Fix DataContext templates test (#4678)
- [BUGFIX] update module_name in NoteBookConfigSchema from v2 path to v3 (#4589) (thanks @Josephmaclean)
- [BUGFIX] request S3 bucket location only when necessary (#4526) (thanks @error418)
- [DOCS] Update
ignored_columns
snippet in "Getting Started" (#4609) - [DOCS] Fixes import statement. (#4694)
- [DOCS] Update tutorial_review.md typo with intended word. (#4611) (thanks @cjbramble)
- [DOCS] Correct typo in url in docstring for set_based_column_map_expectation_template.py (example script) (#4817)
- [MAINTENANCE] Add retries to
requests
in usage stats integration tests (#4600) - [MAINTENANCE] Miscellaneous test cleanup (#4602)
- [MAINTENANCE] Simplify ParameterBuilder.build_parameter() interface (#4622)
- [MAINTENANCE] War on Warnings - DataContext (#4572)
- [MAINTENANCE] Update links within great_expectations.yml (#4549) (thanks @andyjessen)
- [MAINTENANCE] Provide cardinality limit modes from CategoricalColumnDomainBuilder (#4662)
- [MAINTENANCE] Rule-Based Profiler: Rename Rule.generate() to Rule.run() (#4670)
- [MAINTENANCE] Refactor ValidationParameter computation (to be more elegant/compact) and fix a type hint in SimpleDateFormatStringParameterBuilder (#4687)
- [MAINTENANCE] Remove
pybigquery
check that is no longer needed (#4681) - [MAINTENANCE] Rule-Based Profiler: Allow ExpectationConfigurationBuilder to be Optional (#4698)
- [MAINTENANCE] Slightly Clean Up NumericMetricRangeMultiBatchParameterBuilder (#4699)
- [MAINTENANCE] ParameterBuilder must not recompute its value, if it already exists in RuleState (ParameterContainer for its Domain). (#4701)
- [MAINTENANCE] Improve get validator functionality (#4661)
- [MAINTENANCE] Add checks for mostly=1.0 for all renderers (#4736)
- [MAINTENANCE] revert to not raising datasource errors on data context init (#4732)
- [MAINTENANCE] Remove unused bootstrap methods that were migrated to ML Flow (#4742)
- [MAINTENANCE] Update README.md (#4595) (thanks @andyjessen)
- [MAINTENANCE] Check for mostly equals 1 in renderers (#4815)
- [MAINTENANCE] Remove bootstrap tests that are no longer needed (#4818)
- [HACKATHON] ExpectColumnValuesToBeIsoLanguages (#4627) (thanks @szecsip)
- [HACKATHON] ExpectColumnAverageLatLonPairwiseDistanceToBeLessThan (#4559) (thanks @mmi333)
- [HACKATHON] ExpectColumnValuesToBeValidIPv6 (#4561) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidMac (#4562) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidMIME (#4563) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidHexColor (#4564) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidIban (#4565) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidIsoCountry (#4566) (thanks @voidforall)
- [HACKATHON] add expect_column_values_to_be_private_ipv4_class (#4656) (thanks @szecsip)
- [HACKATHON] Feature/expect column values url hostname match with cert (#4649) (thanks @szecsip)
- [HACKATHON] add expect_column_values_url_has_got_valid_cert (#4648) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_valid_us_state_or_territory (#4655) (thanks @Derekma73)
- [HACKATHON] ExpectColumnValuesToBeValidSsn (#4646) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidHttpStatusName (#4645) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidHttpStatusCode (#4644) (thanks @voidforall)
- [HACKATHON] Feature/expect column values to be daytime (#4643) (thanks @szecsip)
- [HACKATHON] add expect_column_values_ip_address_in_network (#4640) (thanks @szecsip)
- [HACKATHON] add expect_column_values_ip_asn_country_code_in_set (#4638) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_valid_us_state (#4654) (thanks @Derekma73)
- [HACKATHON] add expect_column_values_to_be_valid_us_state_or_territory_abbreviation (#4653) (thanks @Derekma73)
- [HACKATHON] add expect_column_values_to_be_weekday (#4636) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_valid_us_state_abbrevation (#4650) (thanks @Derekma73)
- [HACKATHON] ExpectColumnValuesGeometryDistanceToAddressToBeBetween (#4652) (thanks @pjdobson)
- [HACKATHON] ExpectColumnValuesToBeValidUdpPort (#4635) (thanks @voidforall)
- [HACKATHON] add expect_column_values_to_be_fibonacci_number (#4629) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_slug (#4628) (thanks @szecsip)
- [HACKATHON] ExpectColumnValuesGeometryToBeWithinPlace (#4626) (thanks @pjdobson)
- [HACKATHON] add expect_column_values_to_be_private_ipv6 (#4624) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_private_ip_v4 (#4623) (thanks @szecsip)
- [HACKATHON] ExpectColumnValuesToBeValidPrice (#4593) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidPhonenumber (#4592) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBePolygonAreaBetween (#4591) (thanks @mmi333)
- [HACKATHON] ExpectColumnValuesToBeValidTcpPort (#4634) (thanks @voidforall)
- [FEATURE] Convert Existing Self-Initializing Expectations to Make ExpectationConfigurationBuilder Self-Contained with its own validation_parameter_builder settings (#4547)
- [FEATURE] Improve diagnostic checklist details (#4548)
- [BUGFIX] Moves testing dependencies out of core reqs (#4522)
- [BUGFIX] Adjust output of datetime
ParameterBuilder
to match Expectation (#4590) - [DOCS] Technical term tags for Adding features to Expectations section of the ToC (#4462)
- [DOCS] Contributing integrations ToC update. (#4551)
- [DOCS] Update intro page overview image (#4540)
- [DOCS] clarifications on execution engines and scalability (#4539)
- [DOCS] technical terms for validate data advanced (#4535)
- [DOCS] technical terms for validate data actions docs (#4518)
- [DOCS] correct code reference line numbers and snippet tags for how to create a batch of data from an in memory data frame (#4573)
- [DOCS] Update links in page; fix markdown link in html block (#4585)
- [MAINTENANCE] Don't return from validate configuration methods (#4545)
- [MAINTENANCE] Rule-Based Profiler: Refactor utilities into appropriate modules/classes for better separation of concerns (#4553)
- [MAINTENANCE] Refactor global
conftest
(#4534) - [MAINTENANCE] clean up docstrings (#4554)
- [MAINTENANCE] Small formatting rearrangement for RegexPatternStringParameterBuilder (#4558)
- [MAINTENANCE] Refactor Anonymizer utilizing the Strategy design pattern (#4485)
- [MAINTENANCE] Remove duplicate
mistune
dependency (#4569) - [MAINTENANCE] Run PEP273 checks on a schedule or release cut (#4570)
- [MAINTENANCE] Package dependencies usage stats instrumentation - part 1 (#4546)
- [MAINTENANCE] Add DevRel team to GitHub auto-label action (#4575)
- [MAINTENANCE] Add GitHub action to conditionally auto-update PR's (#4574)
- [MAINTENANCE] Bump version of
black
in response to hotfix for Click v8.1.0 (#4577) - [MAINTENANCE] Update overview.md (#4556)
- [MAINTENANCE] Minor clean-up (#4571)
- [MAINTENANCE] Instrument package dependencies (#4583)
- [MAINTENANCE] Standardize DomainBuilder Constructor Arguments Ordering (#4599)
- [FEATURE] Enables Regex-Based Column Map Expectations (#4315)
- [FEATURE] Update diagnostic checklist to do linting checks (#4491)
- [FEATURE] format docstrings as markdown for gallery (#4502)
- [FEATURE] Introduces SetBasedColumnMapExpectation w/ supporting templates & doc (#4497)
- [FEATURE]
YAMLHandler
Class (#4510) - [FEATURE] Remove conflict between filter directives and row_conditions (#4488)
- [FEATURE] Add SNS as a Validation Action (#4519) (thanks @michael-j-thomas)
- [BUGFIX] Fixes ExpectColumnValuesToBeInSet to enable behavior indicated in Parameterized Expectations Doc (#4455)
- [BUGFIX] Fixes minor typo in custom expectation docs, adds missing link (#4507)
- [BUGFIX] Removes validate_config from RegexBasedColumnMap templates & doc (#4506)
- [BUGFIX] Update ExpectColumnValuesToMatchRegex to support parameterized expectations (#4504)
- [BUGFIX] Add back
nbconvert
to dev dependencies (#4515) - [BUGFIX] Account for case where SQLAlchemy dialect is not downloaded when masking a given URL (#4516)
- [BUGFIX] Fix failing test for
How to Configure Credentials
(#4525) - [BUGFIX] Remove Temp Dir (#4528)
- [BUGFIX] Add pin to Jinja 2 due to API changes in v3.1.0 release (#4537)
- [BUGFIX] Fixes broken links in How To Write A How-To Guide (#4536)
- [BUGFIX] Removes cryptography upper bound for general reqs (#4487)
- [BUGFIX] Don't assume boto3 is installed (#4542)
- [DOCS] Update tutorial_review.md (#3981)
- [DOCS] Update AUTHORING_INTRO.md (#4470) (thanks @andyjessen)
- [DOCS] Add clarification (#4477) (thanks @strickvl)
- [DOCS] Add missing word and fix wrong dataset reference (#4478) (thanks @strickvl)
- [DOCS] Adds documentation on how to use Great Expectations with Prefect (#4433) (thanks @desertaxle)
- [DOCS] technical terms validate data checkpoints (#4486)
- [DOCS] How to use a Custom Expectation (#4467)
- [DOCS] Technical Terms for Validate Data: Overview and Core Skills docs (#4465)
- [DOCS] technical terms create expectations advanced skills (#4441)
- [DOCS] Integration documentation (#4483)
- [DOCS] Adding Meltano implementation pattern to docs (#4509) (thanks @pnadolny13)
- [DOCS] Update tutorial_create_expectations.md (#4512) (thanks @andyjessen)
- [DOCS] Fix relative links on github (#4479) (thanks @andyjessen)
- [DOCS] Update README.md (#4533) (thanks @andyjessen)
- [HACKATHON] ExpectColumnValuesToBeValidIPv4 (#4457) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidIanaTimezone (#4532) (thanks @lucasasmith)
- [MAINTENANCE] Clean up
Checkpoints
documentation and addsnippet
(#4474) - [MAINTENANCE] Finalize Great Expectations contrib JSON structure (#4482)
- [MAINTENANCE] Update expectation filenames to match snake_case of their defined Expectations (#4484)
- [MAINTENANCE] Clean Up Types and Rely on "to_json_dict()" where appropriate (#4489)
- [MAINTENANCE] type hints for Batch Request to be string (which leverages parameter/variable resolution) (#4494)
- [MAINTENANCE] Insure consistent ordering of arguments to ParameterBuilder instantiations (#4496)
- [MAINTENANCE] Refactor build_gallery.py script (#4493)
- [MAINTENANCE] Feature/cloud 385/mask cloud creds (#4444)
- [MAINTENANCE] Enforce consistent JSON schema through usage stats (#4499)
- [MAINTENANCE] Applies
camel_to_snake
util toRegexBasedColumnMapExpectation
(#4511) - [MAINTENANCE] Removes unused dependencies (#4508)
- [MAINTENANCE] Revert changes made to dependencies in #4508 (#4520)
- [MAINTENANCE] Add
compatability
stage todependency_graph
pipeline (#4514) - [MAINTENANCE] Add prod metadata and remove package attribute from library_metadata (#4517)
- [MAINTENANCE] Move builder instantiation methods to utility module for broader usage among sub-components within Rule-Based Profiler (#4524)
- [MAINTENANCE] Update package info for Capital One DataProfiler (#4523)
- [MAINTENANCE] Remove tag 'needs migration to modular expectations api' for some Expectations (#4521)
- [MAINTENANCE] Add type hints and PyCharm macros in a test module for DefaultExpectationConfigurationBuilder (#4529)
- [MAINTENANCE] Continue War on Warnings (#4500)
- [FEATURE] Script to validate docs snippets line number refs (#4377)
- [FEATURE] GitHub action to auto label
core-team
(#4382) - [FEATURE]
add_rule()
method for RuleBasedProfilers and tests (#4358) - [FEATURE] Enable the passing of an existing suite to
RuleBasedProfiler.run()
(#4386) - [FEATURE] Impose Ordering on Marshmallow Schema validated Rule-Based Profiler Configuration fields (#4388)
- [FEATURE] Use more granular requirements-dev-xxx.txt files (#4327)
- [FEATURE] Rule-Based Profiler: Implement Utilities for getting all available parameter node names and objects resident in memory (#4442)
- [BUGFIX] Minor Serialization Correction for MeanUnexpectedMapMetricMultiBatchParameterBuilder (#4385)
- [BUGFIX] Fix CategoricalColumnDomainBuilder to be compliant with serialization / instantiation interfaces (#4395)
- [BUGFIX] Fix bug around
get_parent
usage stats utility intest_yaml_config
(#4410) - [BUGFIX] Adding
--spark
flag back toazure-pipelines.yml
compatibility_matrix stage. (#4418) - [BUGFIX] Remove remaining usage of --no-spark and --no-postgresql flags for pytest (#4425)
- [BUGFIX] Insure Proper Indexing of Metric Computation Results in ParameterBuilder (#4426)
- [BUGFIX] Include requirements-dev-contrib.txt in dev-install-matrix.yml for lightweight (#4430)
- [BUGFIX] Remove
pytest-azurepiplines
usage fromtest_cli
stages in Azure pipelines (#4432) - [BUGFIX] Updates or deletes broken and deprecated example notebooks (#4404)
- [BUGFIX] Add any dependencies we import directly, but don't have as explicit requirements (#4447)
- [BUGFIX] Removes potentially sensitive webhook URLs from logging (#4440)
- [BUGFIX] Fix packaging test (#4452)
- [DOCS] Fix typo in how_to_create_custom_metrics (#4379)
- [DOCS] Add
snippet
tag to gcs data docs (#4383) - [DOCS] adjust lines for py reference (#4390)
- [DOCS] technical tags for connecting to data: core skills docs (#4403)
- [DOCS] technical term tags for connect to data database documents (#4413)
- [DOCS] Technical term tags for documentation under Connect to data: Filesystem (#4411)
- [DOCS] Technical term tags for setup pages (#4392)
- [DOCS] Technical term tags for Connect to Data: Advanced docs. (#4406)
- [DOCS] Technical tags: Connect to data:In memory docs (#4405)
- [DOCS] Add misc
snippet
tags to existing documentation (#4397) - [DOCS] technical terms create expectations: core skills (#4435)
- [DOCS] Creates Custom Table Expectation How-To (#4399)
- [HACKATHON] ExpectTableLinearFeatureImportancesToBe (#4400)
- [MAINTENANCE] Group MAP_SERIES and MAP_CONDITION_SERIES with VALUE-type metrics (#3286)
- [MAINTENANCE] minor imports cleanup (#4381)
- [MAINTENANCE] Change schedule for
packaging_and_installation
pipeline to run at off-hours (#4384) - [MAINTENANCE] Implicitly anonymize object based on module (#4387)
- [MAINTENANCE] Preparatory cleanup refactoring of get_compute_domain (#4371)
- [MAINTENANCE] RBP -- make parameter builder configurations for self initializing expectations consistent with ParameterBuilder class interfaces (#4398)
- [MAINTENANCE] Refactor
ge_class
attr out of Anonymizer and related child classes (#4393) - [MAINTENANCE] Removing Custom Expectation Renderer docs from sidebar (#4401)
- [MAINTENANCE] Enable "rule_based_profiler.run()" Method to Accept Batch Data Arguments Directly (#4409)
- [MAINTENANCE] Refactor out unnecessary Anonymizer child classes (#4408)
- [MAINTENANCE] Replace "sampling_method" with "estimator" in Rule-Based Profiler code (#4420)
- [MAINTENANCE] Add docstrings and type hints to
Anonymizer
(#4419) - [MAINTENANCE] Continue chipping away at warnings (#4422)
- [MAINTENANCE] Rule-Based Profiler: Standardize on Include/Exclude Column Names List (#4424)
- [MAINTENANCE] Set upper bound on number of allowed warnings in snippet validation script (#4434)
- [MAINTENANCE] Clean up of
RegexPatternStringParameterBuilder
tests to use unittests (#4436)
- [FEATURE] ParameterBuilder for Computing Average Unexpected Values Fractions for any Map Metric (#4340)
- [FEATURE] Improve bootstrap quantile method accuracy (#4270)
- [FEATURE] Decorate RuleBasedProfiler.run() with usage statistics (#4321)
- [FEATURE] MapMetricColumnDomainBuilder for Rule-Based Profiler (#4353)
- [FEATURE] Enable expect_column_min/_max_to_be_between expectations to be self-initializing (#4363)
- [FEATURE] Azure pipeline to perform nightly CI/CD runs around packaging/installation (#4274)
- [BUGFIX] Fix
IndexError
around data asset pagination from CLI (#4346) - [BUGFIX] Upper bound pyathena to <2.5.0 (#4350)
- [BUGFIX] Fixes PyAthena type checking for core expectations & tests (#4359)
- [BUGFIX] BatchRequest serialization (CLOUD-743) (#4352)
- [BUGFIX] Update the favicon on docs site (#4376)
- [BUGFIX] Fix issue with datetime objects in expecatation args (#2652) (thanks @jstammers)
- [DOCS] Universal map TOC update (#4292)
- [DOCS] add Config section (#4355)
- [DOCS] Deployment Patterns to Reference Architectures (#4344)
- [DOCS] Fixes tutorial link in reference architecture prereqs component (#4360)
- [DOCS] Tag technical terms in getting started tutorial (#4354)
- [DOCS] Update overview pages to link to updated tutorial pages. (#4378)
- [HACKATHON] ExpectColumnValuesToBeValidUUID (#4322)
- [HACKATHON] add expectation core (#4357)
- [HACKATHON] ExpectColumnAverageToBeWithinRangeOfGivenPoint (#4356)
- [MAINTENANCE] rule based profiler minor clean up of ValueSetParameterBuilder (#4332)
- [MAINTENANCE] Adding tests that exercise single and multi-batch BatchRequests (#4330)
- [MAINTENANCE] Formalize ParameterBuilder contract API usage in ValueSetParameterBuilder (#4333)
- [MAINTENANCE] Rule-Based Profiler: Create helpers directory; use column domain generation convenience method (#4335)
- [MAINTENANCE] Deduplicate table domain kwargs splitting (#4338)
- [MAINTENANCE] Update Azure CI/CD cron schedule to run more frequently (#4345)
- [MAINTENANCE] Optimize CategoricalColumnDomainBuilder to compute metrics in a single method call (#4348)
- [MAINTENANCE] Reduce tries to 2 for probabilistic tests (#4351)
- [MAINTENANCE] Refactor Checkpoint toolkit (#4342)
- [MAINTENANCE] Refactor all uses of
format
in favor of f-strings (#4347) - [MAINTENANCE] Update great_expectations_contrib CLI tool to use existing diagnostic classes (#4316)
- [MAINTENANCE] Setting stage for removal of
--no-postgresql
and--no-spark
flags frompytest
. Enable--postgresql
and--spark
(#4309) - [MAINTENANCE] convert unexpected_list contents to hashable type (#4336)
- [MAINTENANCE] add operator and func handling to stores urns (#4334)
- [MAINTENANCE] Refactor ParameterBuilder classes to extend parent class where possible; also, minor cleanup (#4375)
- [FEATURE] Enable Simultaneous Execution of all Metric Computations for ParameterBuilder implementations in Rule-Based Profiler (#4282)
- [FEATURE] Update print_diagnostic_checklist with an option to show any failed tests (#4288)
- [FEATURE] Self-Initializing Expectations (implemented for three example expectations). (#4258)
- [FEATURE] ValueSetMultiBatchParameterBuilder and CategoricalColumnDomainBuilder (#4269)
- [FEATURE] Remove changelog-bot GitHub Action (#4297)
- [FEATURE] Add requirements-dev-lite.txt and update tests/docs (#4273)
- [FEATURE] Enable All ParameterBuilder and DomainBuilder classes to accept batch_list generically (#4302)
- [FEATURE] Enable Probabilistic Tests To Retry upon Assertion Failure (#4308)
- [FEATURE] Update usage stats schema to account for RBP's run() payload (#4266)
- [FEATURE] ProfilerRunAnonymizer (#4264)
- [FEATURE] Enable Expectation "expect_column_values_to_be_in_set" to be Self-Initializing (#4318)
- [BUGFIX] Add redirect for removed Spark EMR page (#4280)
- [BUGFIX]
ConfiguredAssetSqlDataConnector
now correctly handlesschema
andprefix
/suffix
(#4268) - [BUGFIX] Fixes Expectation Diagnostics failing on multi-line docstrings with leading linebreaks (#4286)
- [BUGFIX] Respect test backends (#4287)
- [BUGFIX] Skip test__generate_expectations_tests__xxx tests when sqlalchemy isn't there (#4300)
- [BUGFIX] test_backends integration test fix and supporting docs code ref fixes (#4306)
- [BUGFIX] Update
deep_filter_properties_iterable
to ensure that empty values are cleaned (#4298) - [BUGFIX] Fixes validate_configuration checking in diagnostics (#4307)
- [BUGFIX] Update test output that should be returned from generate_diagnostic_checklist (#4317)
- [BUGFIX] Standardizes imports in expectation templates and examples (#4320)
- [BUGFIX] Only validate row_condition if not None (#4329)
- [BUGFIX] Fix PEP273 Windows issue (#4328)
- [DOCS] Fixes misc. verbiage & typos in new Custom Expectation docs (#4283)
- [DOCS] fix formatting in configuration details block of Getting Started (#4289) (thanks @afeld)
- [DOCS] Fixes imports and code refs to expectation templates (#4314)
- [DOCS] Update creating_custom_expectations/overview.md (#4278) (thanks @binarytom)
- [CONTRIB] CapitalOne Dataprofiler expectations (#4174) (thanks @taylorfturner)
- [HACKATHON] ExpectColumnValuesToBeLatLonCoordinatesInRangeOfGivenPoint (#4284)
- [HACKATHON] ExpectColumnValuesToBeValidDegreeDecimalCoordinates (#4319)
- [MAINTENANCE] Refactor parameter setting for simpler ParameterBuilder interface (#4299)
- [MAINTENANCE] SimpleDateTimeFormatStringParameterBuilder and general RBP example config updates (#4304)
- [MAINTENANCE] Make adherence to Marshmallow Schema more robust (#4325)
- [MAINTENANCE] Refactor rule based profiler to keep objects/utilities within intended scope (#4331)
- [MAINTENANCE] Dependabot version upgrades (#4253, #4231, #4058, #4041, #3916, #3886, #3583, #2856, #3370, #3216, #2935, #2855, #3302, #4008, #4252)
- [FEATURE] Add
run_profiler_on_data
method to DataContext (#4190) - [FEATURE]
RegexPatternStringParameterBuilder
forRuleBasedProfiler
(#4167) - [FEATURE] experimental column map expectation checking for vectors (#3102) (thanks @manyshapes)
- [FEATURE] Pre-requisites in Rule-Based Profiler for Self-Estimating Expectations (#4242)
- [FEATURE] Add optional parameter
condition
to DefaultExpectationConfigurationBuilder (#4246) - [BUGFIX] Ensure that test result for
RegexPatternStringParameterBuilder
is deterministic (#4240) - [BUGFIX] Remove duplicate RegexPatternStringParameterBuilder test (#4241)
- [BUGFIX] Improve pandas version checking in test_expectations[_cfe].py files (#4248)
- [BUGFIX] Ensure
test_script_runner.py
actually raises AssertionErrors correctly (#4239) - [BUGFIX] Check for pandas>=024 not pandas>=24 (#4263)
- [BUGFIX] Add support for SqlAlchemyQueryStore connection_string credentials (#4224) (thanks @davidvanrooij)
- [BUGFIX] Remove assertion (#4271)
- [DOCS] Hackathon Contribution Docs (#3897)
- [MAINTENANCE] Rule-Based Profiler: Fix Circular Imports; Configuration Schema Fixes; Enhanced Unit Tests; Pre-Requisites/Refactoring for Self-Estimating Expectations (#4234)
- [MAINTENANCE] Reformat contrib expectation with black (#4244)
- [MAINTENANCE] Resolve cyclic import issue with usage stats (#4251)
- [MAINTENANCE] Additional refactor to clean up cyclic imports in usage stats (#4256)
- [MAINTENANCE] Rule-Based Profiler prerequisite: fix quantiles profiler configuration and add comments (#4255)
- [MAINTENANCE] Introspect Batch Request Dictionary for its kind and instantiate accordingly (#4259)
- [MAINTENANCE] Minor clean up in style of an RBP test fixture; making variables access more robust (#4261)
- [MAINTENANCE] define empty sqla_bigquery object (#4249)
- [FEATURE] Support Multi-Dimensional Metric Computations Generically for Multi-Batch Parameter Builders (#4206)
- [FEATURE] Add support for sqlalchemy-bigquery while falling back on pybigquery (#4182)
- [BUGFIX] Update validate_configuration for core Expectations that don't return True (#4216)
- [DOCS] Fixes two references to the Getting Started tutorial (#4189)
- [DOCS] Deepnote Deployment Pattern Guide (#4169)
- [DOCS] Allow Data Docs to be rendered in night mode (#4130)
- [DOCS] Fix datepicker filter on data docs (#4217)
- [DOCS] Deepnote Deployment Pattern Image Fixes (#4229)
- [MAINTENANCE] Refactor RuleBasedProfiler toolkit pattern (#4191)
- [MAINTENANCE] Revert
dependency_graph
pipeline changes to ensureusage_stats
runs in parallel (#4198) - [MAINTENANCE] Refactor relative imports (#4195)
- [MAINTENANCE] Remove temp file that was accidently committed (#4201)
- [MAINTENANCE] Update default candidate strings SimpleDateFormatString parameter builder (#4193)
- [MAINTENANCE] minor type hints clean up (#4214)
- [MAINTENANCE] RBP testing framework changes (#4184)
- [MAINTENANCE] add conditional check for 'expect_column_values_to_be_in_type_list' (#4200)
- [MAINTENANCE] Allow users to pass in any set of polygon points in expectation for point to be within region (#2520) (thanks @ryanlindeborg)
- [MAINTENANCE] Better support Hive, better support BigQuery. (#2624) (thanks @jacobpgallagher)
- [MAINTENANCE] move process_evaluation_parameters into conditional (#4109)
- [MAINTENANCE] Type hint usage stats (#4226)
- [FEATURE] Create profiler from DataContext (#4070)
- [FEATURE] Add read_sas function (#3972) (thanks @andyjessen)
- [FEATURE] Run profiler from DataContext (#4141)
- [FEATURE] Instantiate Rule-Based Profiler Using Typed Configuration Object (#4150)
- [FEATURE] Provide ability to instantiate Checkpoint using CheckpointConfig typed object (#4166)
- [FEATURE] Misc cleanup around CLI
suite
command and related utilities (#4158) - [FEATURE] Add scheduled runs for primary Azure pipeline (#4117)
- [FEATURE] Promote dependency graph test strategy to production (#4124)
- [BUGFIX] minor updates to test definition json files (#4123)
- [BUGFIX] Fix typo for metric name in expect_column_values_to_be_edtf_parseable (#4140)
- [BUGFIX] Ensure that CheckpointResult object can be pickled (#4157)
- [BUGFIX] Custom notebook templates (#2619) (thanks @luke321321)
- [BUGFIX] Include public fields in property_names (#4159)
- [DOCS] Reenable docs-under-test for RuleBasedProfiler (#4149)
- [DOCS] Provided details for using GE_HOME in commandline. (#4164)
- [MAINTENANCE] Return Rule-Based Profiler base.py to its dedicated config subdirectory (#4125)
- [MAINTENANCE] enable filter properties dict to handle both inclusion and exclusion lists (#4127)
- [MAINTENANCE] Remove unused Great Expectations imports (#4135)
- [MAINTENANCE] Update trigger for scheduled Azure runs (#4134)
- [MAINTENANCE] Maintenance/upgrade black (#4136)
- [MAINTENANCE] Alter
great_expectations
pipeline trigger to be more consistent (#4138) - [MAINTENANCE] Remove remaining unused imports (#4137)
- [MAINTENANCE] Remove
class_name
as mandatory field fromRuleBasedProfiler
(#4139) - [MAINTENANCE] Ensure
AWSAthena
does not create temporary table as part of processing Batch by default, which is currently not supported (#4103) - [MAINTENANCE] Remove unused
Exception as e
instances (#4143) - [MAINTENANCE] Standardize DictDot Method Behaviors Formally for Consistent Usage Patterns in Subclasses (#4131)
- [MAINTENANCE] Remove unused f-strings (#4142)
- [MAINTENANCE] Minor Validator code clean up -- for better code clarity (#4147)
- [MAINTENANCE] Refactoring of
test_script_runner.py
. Integration and Docs tests (#4145) - [MAINTENANCE] Remove
compatability
stage fromdependency-graph
pipeline (#4161) - [MAINTENANCE] CLOUD-618: GE Cloud "account" to "organization" rename (#4146)
- [FEATURE] Delete profilers from DataContext (#4067)
- [FEATURE] [BUGFIX] Support nullable int column types (#4044) (thanks @scnerd)
- [FEATURE] Rule-Based Profiler Configuration and Runtime Arguments Reconciliation Logic (#4111)
- [BUGFIX] Add default BIGQUERY_TYPES (#4096)
- [BUGFIX] Pin
pip --upgrade
to a specific version for CI/CD pipeline (#4100) - [BUGFIX] Use
pip==20.2.4
for usage statistics stage of CI/CD (#4102) - [BUGFIX] Fix shared state issue in renderer test (#4000)
- [BUGFIX] Missing docstrings on validator expect_ methods (#4062) (#4081)
- [BUGFIX] Fix s3 path suffix bug on windows (#4042) (thanks @scnerd)
- [MAINTENANCE] fix typos in changelogs (#4093)
- [MAINTENANCE] Migration of GCP tests to new project (#4072)
- [MAINTENANCE] Refactor Validator methods (#4095)
- [MAINTENANCE] Fix Configuration Schema and Refactor Rule-Based Profiler; Initial Implementation of Reconciliation Logic Between Configuration and Runtime Arguments (#4088)
- [MAINTENANCE] Minor Cleanup -- remove unnecessary default arguments from dictionary cleaner (#4110)
- [BUGFIX] Fix typing_extensions requirement to allow for proper build (#4083) (thanks @vojtakopal and @Godoy)
- [DOCS] data docs action rewrite (#4087)
- [DOCS] metric store how to rewrite (#4086)
- [MAINTENANCE] Change
logger.warn
tologger.warning
to remove deprecation warnings (#4085)
- [FEATURE] Profiler Store (#3990)
- [FEATURE] List profilers from DataContext (#4023)
- [FEATURE] add bigquery json credentials kwargs for sqlalchemy connect (#4039)
- [FEATURE] Get profilers from DataContext (#4033)
- [FEATURE] Add RuleBasedProfiler to
test_yaml_config
utility (#4038) - [BUGFIX] Checkpoint Configurator fix to allow notebook logging suppression (#4057)
- [DOCS] Created a page containing our glossary of terms and definitions. (#4056)
- [DOCS] swap of old uri for new in data docs generated (#4013)
- [MAINTENANCE] Refactor
test_yaml_config
(#4029) - [MAINTENANCE] Additional distinction made between V2 and V3 upgrade script (#4046)
- [MAINTENANCE] Correcting Checkpoint Configuration and Execution Implementation (#4015)
- [MAINTENANCE] Update minimum version for SQL Alchemy (#4055)
- [MAINTENANCE] Refactor RBP constructor to work with **kwargs instantiation pattern through config objects (#4043)
- [MAINTENANCE] Remove unnecessary metric dependency evaluations and add common table column types metric. (#4063)
- [MAINTENANCE] Clean up new RBP types, method signatures, and method names for the long term. (#4064)
- [MAINTENANCE] fixed broken function call in CLI (#4068)
- [FEATURE] Add
run_profiler_on_data
method to DataContext (#4190) - [FEATURE]
RegexPatternStringParameterBuilder
forRuleBasedProfiler
(#4167) - [FEATURE] experimental column map expectation checking for vectors (#3102) (thanks @manyshapes)
- [FEATURE] Pre-requisites in Rule-Based Profiler for Self-Estimating Expectations (#4242)
- [FEATURE] Add optional parameter
condition
to DefaultExpectationConfigurationBuilder (#4246) - [BUGFIX] Ensure that test result for
RegexPatternStringParameterBuilder
is deterministic (#4240) - [BUGFIX] Remove duplicate RegexPatternStringParameterBuilder test (#4241)
- [BUGFIX] Improve pandas version checking in test_expectations[_cfe].py files (#4248)
- [BUGFIX] Ensure
test_script_runner.py
actually raises AssertionErrors correctly (#4239) - [BUGFIX] Check for pandas>=024 not pandas>=24 (#4263)
- [BUGFIX] Add support for SqlAlchemyQueryStore connection_string credentials (#4224) (thanks @davidvanrooij)
- [BUGFIX] Remove assertion (#4271)
- [DOCS] Hackathon Contribution Docs (#3897)
- [MAINTENANCE] Rule-Based Profiler: Fix Circular Imports; Configuration Schema Fixes; Enhanced Unit Tests; Pre-Requisites/Refactoring for Self-Estimating Expectations (#4234)
- [MAINTENANCE] Reformat contrib expectation with black (#4244)
- [MAINTENANCE] Resolve cyclic import issue with usage stats (#4251)
- [MAINTENANCE] Additional refactor to clean up cyclic imports in usage stats (#4256)
- [MAINTENANCE] Rule-Based Profiler prerequisite: fix quantiles profiler configuration and add comments (#4255)
- [MAINTENANCE] Introspect Batch Request Dictionary for its kind and instantiate accordingly (#4259)
- [MAINTENANCE] Minor clean up in style of an RBP test fixture; making variables access more robust (#4261)
- [MAINTENANCE] define empty sqla_bigquery object (#4249)
- [FEATURE] Support Multi-Dimensional Metric Computations Generically for Multi-Batch Parameter Builders (#4206)
- [FEATURE] Add support for sqlalchemy-bigquery while falling back on pybigquery (#4182)
- [BUGFIX] Update validate_configuration for core Expectations that don't return True (#4216)
- [DOCS] Fixes two references to the Getting Started tutorial (#4189)
- [DOCS] Deepnote Deployment Pattern Guide (#4169)
- [DOCS] Allow Data Docs to be rendered in night mode (#4130)
- [DOCS] Fix datepicker filter on data docs (#4217)
- [DOCS] Deepnote Deployment Pattern Image Fixes (#4229)
- [MAINTENANCE] Refactor RuleBasedProfiler toolkit pattern (#4191)
- [MAINTENANCE] Revert
dependency_graph
pipeline changes to ensureusage_stats
runs in parallel (#4198) - [MAINTENANCE] Refactor relative imports (#4195)
- [MAINTENANCE] Remove temp file that was accidently committed (#4201)
- [MAINTENANCE] Update default candidate strings SimpleDateFormatString parameter builder (#4193)
- [MAINTENANCE] minor type hints clean up (#4214)
- [MAINTENANCE] RBP testing framework changes (#4184)
- [MAINTENANCE] add conditional check for 'expect_column_values_to_be_in_type_list' (#4200)
- [MAINTENANCE] Allow users to pass in any set of polygon points in expectation for point to be within region (#2520) (thanks @ryanlindeborg)
- [MAINTENANCE] Better support Hive, better support BigQuery. (#2624) (thanks @jacobpgallagher)
- [MAINTENANCE] move process_evaluation_parameters into conditional (#4109)
- [MAINTENANCE] Type hint usage stats (#4226)
- [FEATURE] Create profiler from DataContext (#4070)
- [FEATURE] Add read_sas function (#3972) (thanks @andyjessen)
- [FEATURE] Run profiler from DataContext (#4141)
- [FEATURE] Instantiate Rule-Based Profiler Using Typed Configuration Object (#4150)
- [FEATURE] Provide ability to instantiate Checkpoint using CheckpointConfig typed object (#4166)
- [FEATURE] Misc cleanup around CLI
suite
command and related utilities (#4158) - [FEATURE] Add scheduled runs for primary Azure pipeline (#4117)
- [FEATURE] Promote dependency graph test strategy to production (#4124)
- [BUGFIX] minor updates to test definition json files (#4123)
- [BUGFIX] Fix typo for metric name in expect_column_values_to_be_edtf_parseable (#4140)
- [BUGFIX] Ensure that CheckpointResult object can be pickled (#4157)
- [BUGFIX] Custom notebook templates (#2619) (thanks @luke321321)
- [BUGFIX] Include public fields in property_names (#4159)
- [DOCS] Reenable docs-under-test for RuleBasedProfiler (#4149)
- [DOCS] Provided details for using GE_HOME in commandline. (#4164)
- [MAINTENANCE] Return Rule-Based Profiler base.py to its dedicated config subdirectory (#4125)
- [MAINTENANCE] enable filter properties dict to handle both inclusion and exclusion lists (#4127)
- [MAINTENANCE] Remove unused Great Expectations imports (#4135)
- [MAINTENANCE] Update trigger for scheduled Azure runs (#4134)
- [MAINTENANCE] Maintenance/upgrade black (#4136)
- [MAINTENANCE] Alter
great_expectations
pipeline trigger to be more consistent (#4138) - [MAINTENANCE] Remove remaining unused imports (#4137)
- [MAINTENANCE] Remove
class_name
as mandatory field fromRuleBasedProfiler
(#4139) - [MAINTENANCE] Ensure
AWSAthena
does not create temporary table as part of processing Batch by default, which is currently not supported (#4103) - [MAINTENANCE] Remove unused
Exception as e
instances (#4143) - [MAINTENANCE] Standardize DictDot Method Behaviors Formally for Consistent Usage Patterns in Subclasses (#4131)
- [MAINTENANCE] Remove unused f-strings (#4142)
- [MAINTENANCE] Minor Validator code clean up -- for better code clarity (#4147)
- [MAINTENANCE] Refactoring of
test_script_runner.py
. Integration and Docs tests (#4145) - [MAINTENANCE] Remove
compatability
stage fromdependency-graph
pipeline (#4161) - [MAINTENANCE] CLOUD-618: GE Cloud "account" to "organization" rename (#4146)
- [FEATURE] Delete profilers from DataContext (#4067)
- [FEATURE] [BUGFIX] Support nullable int column types (#4044) (thanks @scnerd)
- [FEATURE] Rule-Based Profiler Configuration and Runtime Arguments Reconciliation Logic (#4111)
- [BUGFIX] Add default BIGQUERY_TYPES (#4096)
- [BUGFIX] Pin
pip --upgrade
to a specific version for CI/CD pipeline (#4100) - [BUGFIX] Use
pip==20.2.4
for usage statistics stage of CI/CD (#4102) - [BUGFIX] Fix shared state issue in renderer test (#4000)
- [BUGFIX] Missing docstrings on validator expect_ methods (#4062) (#4081)
- [BUGFIX] Fix s3 path suffix bug on windows (#4042) (thanks @scnerd)
- [MAINTENANCE] fix typos in changelogs (#4093)
- [MAINTENANCE] Migration of GCP tests to new project (#4072)
- [MAINTENANCE] Refactor Validator methods (#4095)
- [MAINTENANCE] Fix Configuration Schema and Refactor Rule-Based Profiler; Initial Implementation of Reconciliation Logic Between Configuration and Runtime Arguments (#4088)
- [MAINTENANCE] Minor Cleanup -- remove unnecessary default arguments from dictionary cleaner (#4110)
- [BUGFIX] Fix typing_extensions requirement to allow for proper build (#4083) (thanks @vojtakopal and @Godoy)
- [DOCS] data docs action rewrite (#4087)
- [DOCS] metric store how to rewrite (#4086)
- [MAINTENANCE] Change
logger.warn
tologger.warning
to remove deprecation warnings (#4085)
- [FEATURE] Profiler Store (#3990)
- [FEATURE] List profilers from DataContext (#4023)
- [FEATURE] add bigquery json credentials kwargs for sqlalchemy connect (#4039)
- [FEATURE] Get profilers from DataContext (#4033)
- [FEATURE] Add RuleBasedProfiler to
test_yaml_config
utility (#4038) - [BUGFIX] Checkpoint Configurator fix to allow notebook logging suppression (#4057)
- [DOCS] Created a page containing our glossary of terms and definitions. (#4056)
- [DOCS] swap of old uri for new in data docs generated (#4013)
- [MAINTENANCE] Refactor
test_yaml_config
(#4029) - [MAINTENANCE] Additional distinction made between V2 and V3 upgrade script (#4046)
- [MAINTENANCE] Correcting Checkpoint Configuration and Execution Implementation (#4015)
- [MAINTENANCE] Update minimum version for SQL Alchemy (#4055)
- [MAINTENANCE] Refactor RBP constructor to work with **kwargs instantiation pattern through config objects (#4043)
- [MAINTENANCE] Remove unnecessary metric dependency evaluations and add common table column types metric. (#4063)
- [MAINTENANCE] Clean up new RBP types, method signatures, and method names for the long term. (#4064)
- [MAINTENANCE] fixed broken function call in CLI (#4068)
- [FEATURE] Marshmallow schema for Rule Based Profiler (#3982)
- [FEATURE] Enable Rule-Based Profile Parameter Access To Collection Typed Values (#3998)
- [BUGFIX] Docs integration pipeline bugfix (#3997)
- [BUGFIX] Enables spark-native null filtering (#4004)
- [DOCS] Gtm/cta in docs (#3993)
- [DOCS] Fix incorrect variable name in how_to_configure_an_expectation_store_in_amazon_s3.md (#3971) (thanks @moritzkoerber)
- [DOCS] update custom docs css to add a subtle border around tabbed content (#4001)
- [DOCS] Migration Guide now includes example for Spark data (#3996)
- [DOCS] Revamp Airflow Deployment Pattern (#3963) (thanks @denimalpaca)
- [DOCS] updating redirects to reflect a moved file (#4007)
- [DOCS] typo in gcp + bigquery tutorial (#4018)
- [DOCS] Additional description of Kubernetes Operators in GCP Deployment Guide (#4019)
- [DOCS] Migration Guide now includes example for Databases (#4005)
- [DOCS] Update how to instantiate without a yml file (#3995)
- [MAINTENANCE] Refactor of
test_script_runner.py
to break-up test list (#3987) - [MAINTENANCE] Small refactor for tests that allows DB setup to be done from all tests (#4012)
- [FEATURE] Add pagination/search to CLI batch request listing (#3854)
- [BUGFIX] Safeguard against using V2 API with V3 Configuration (#3954)
- [BUGFIX] Bugfix and refactor for
cloud-db-integration
pipeline (#3977) - [BUGFIX] Fixes breaking typo in expect_column_values_to_be_json_parseable (#3983)
- [BUGFIX] Fixes issue where nested columns could not be addressed properly in spark (#3986)
- [DOCS] How to connect to your data in
mssql
(#3950) - [DOCS] MigrationGuide - Adding note on Migrating Expectation Suites (#3959)
- [DOCS] Incremental Update: The Universal Map's Getting Started Tutorial (#3881)
- [DOCS] Note about creating backup of Checkpoints (#3968)
- [DOCS] Connecting to BigQuery Doc line references fix (#3974)
- [DOCS] Remove RTD snippet about comments/suggestions from Docusaurus docs (#3980)
- [DOCS] Add howto for the OpenLineage validation operator (#3688) (thanks @rossturk)
- [DOCS] Updates to README.md (#3964)
- [DOCS] Update migration guide (#3967)
- [MAINTENANCE] Refactor docs dependency script (#3952)
- [MAINTENANCE] Use Effective SQLAlchemy for Reflection Fallback Logic and SQL Metrics (#3958)
- [MAINTENANCE] Remove outdated scripts (#3953)
- [MAINTENANCE] Add pytest opt to improve collection time (#3976)
- [MAINTENANCE] Refactor
render
method in PageRenderer (#3962) - [MAINTENANCE] Standardize rule based profiler testing directories organization (#3984)
- [MAINTENANCE] Metrics Cleanup (#3989)
- [MAINTENANCE] Refactor
render
method of Content Block Renderer (#3960)
- [BREAKING] Change Default CLI Flag To V3 (#3943)
- [FEATURE] Cloud-399/Cloud-519: Add Cloud Notification Action (#3891)
- [FEATURE]
great_expectations_contrib
CLI tool (#3909) - [FEATURE] Update
dependency_graph
pipeline to usedgtest
CLI (#3912) - [FEATURE] Incorporate updated dgtest CLI tool in experimental pipeline (#3927)
- [FEATURE] Add YAML config option to disable progress bars (#3794)
- [BUGFIX] Fix internal links to docs that may be rendered incorrectly (#3915)
- [BUGFIX] Update SlackNotificationAction to send slack_token and slack_channel to send_slack_notification function (#3873) (thanks @Calvo94)
- [BUGFIX]
CheckDocsDependenciesChanges
to only handle.py
files (#3936) - [BUGFIX] Provide ability to capture schema_name for SQL-based datasources; fix method usage bugs. (#3938)
- [BUGFIX] Ensure that Jupyter Notebook cells convert JSON strings to Python-compliant syntax (#3939)
- [BUGFIX] Cloud-519/cloud notification action return type (#3942)
- [BUGFIX] Fix issue with regex groups in
check_docs_deps
(#3949) - [DOCS] Created link checker, fixed broken links (#3930)
- [DOCS] adding the link checker to the build (#3933)
- [DOCS] Add name to link checker in build (#3935)
- [DOCS] GCP Deployment Pattern (#3926)
- [DOCS] remove v3api flag in documentation (#3944)
- [DOCS] Make corrections in HOWTO Guides for Getting Data from SQL Sources (#3945)
- [DOCS] Tiny doc fix (#3948)
- [MAINTENANCE] Fix breaking change caused by the new version of ruamel.yaml (#3908)
- [MAINTENANCE] Drop extraneous print statement in self_check/util.py. (#3905)
- [MAINTENANCE] Raise exceptions on init in cloud mode (#3913)
- [MAINTENANCE] removing commented requirement (#3920)
- [MAINTENANCE] Patch for atomic renderer snapshot tests (#3918)
- [MAINTENANCE] Remove types/expectations.py (#3928)
- [MAINTENANCE] Tests/test data class serializable dot dict (#3924)
- [MAINTENANCE] Ensure that concurrency is backwards compatible (#3872)
- [MAINTENANCE] Fix issue where meta was not recognized as a kwarg (#3852)
- [FEATURE] PandasExecutionEngine is able to instantiate Google Storage client in Google Cloud Composer (#3896)
- [BUGFIX] Revert change to ExpectationSuite constructor (#3902)
- [MAINTENANCE] SQL statements that are of TextClause type expressed as subqueries (#3899)
- [DOCS] Updates to configuring credentials (#3856)
- [DOCS] Add docs on creating suites with the UserConfigurableProfiler (#3877)
- [DOCS] Update how to configure an expectation store in GCS (#3874)
- [DOCS] Update how to configure a validation result store in GCS (#3887)
- [DOCS] Update how to host and share data docs on GCS (#3889)
- [DOCS] Organize metadata store sidebar category by type of store (#3890)
- [MAINTENANCE]
add_expectation()
inExpectationSuite
supports usage statistics for GE. (#3824) - [MAINTENANCE] Clean up Metrics type usage, SQLAlchemyExecutionEngine and SQLAlchemyBatchData implementation, and SQLAlchemy API usage (#3884)
- [FEATURE] Add support for named groups in data asset regex (#3855)
- [BUGFIX] Fix issue where dependency graph tester picks up non *.py files and add test file (#3830)
- [BUGFIX] Ensure proper exit code for dependency graph script (#3839)
- [BUGFIX] Allows GE to work when installed in a zip file (PEP 273). Fixes issue #3772 (#3798) (thanks @joseignaciorc)
- [BUGFIX] Update conditional for TextClause isinstance check in SQLAlchemyExecutionEngine (#3844)
- [BUGFIX] Fix usage stats events (#3857)
- [BUGFIX] Make ExpectationContext optional and remove when null to ensure backwards compatability (#3859)
- [BUGFIX] Fix sqlalchemy expect_compound_columns_to_be_unique (#3827) (thanks @harperweaver-dox)
- [BUGFIX] Ensure proper serialization of SQLAlchemy Legacy Row (#3865)
- [DOCS] Update migration_guide.md (#3832)
- [MAINTENANCE] Remove the need for DataContext registry in the instrumentation of the Legacy Profiler profiling method. (#3836)
- [MAINTENANCE] Remove DataContext registry (#3838)
- [MAINTENANCE] Refactor cli suite conditionals (#3841)
- [MAINTENANCE] adding hints to stores in data context (#3849)
- [MAINTENANCE] Improve usage stats testing (#3858, #3861)
- [MAINTENANCE] Make checkpoint methods in DataContext pass-through (#3860)
- [MAINTENANCE] Datasource and ExecutionEngine Anonymizers handle missing module_name (#3867)
- [MAINTENANCE] Add logging around DatasourceInitializationError in DataContext (#3846)
- [MAINTENANCE] Use f-string to prevent string concat issue in Evaluation Parameters (#3864)
- [MAINTENANCE] Test for errors / invalid messages in logs & fix various existing issues (#3875)
- [FEATURE] Instrument Runtime DataConnector for Usage Statistics: Add "checkpoint.run" Event Schema (#3797)
- [FEATURE] Add suite creation type field to CLI SUITE "new" and "edit" Usage Statistics events (#3810)
- [FEATURE] [EXPERIMENTAL] Dependency graph based testing strategy and related pipeline (#3738, #3815, #3818)
- [FEATURE] BaseDataContext registry (#3812, #3819)
- [FEATURE] Add usage statistics instrumentation to Legacy UserConfigurableProfiler execution (#3828)
- [BUGFIX] CheckpointConfig.deepcopy() must copy all fields, including the null-valued fields (#3793)
- [BUGFIX] Fix issue where configuration store didn't allow nesting (#3811)
- [BUGFIX] Fix Minor Bugs in and Clean Up UserConfigurableProfiler (#3822)
- [BUGFIX] Ensure proper replacement of nulls in Jupyter Notebooks (#3782)
- [BUGFIX] Fix issue where configuration store didn't allow nesting (#3811)
- [DOCS] Clean up TOC (#3783)
- [DOCS] Update Checkpoint and Actions Reference with testing (#3787)
- [DOCS] Update How to install Great Expectations locally (#3805)
- [DOCS] How to install Great Expectations in a hosted environment (#3808)
- [MAINTENANCE] Make BatchData Serialization More Robust (#3791)
- [MAINTENANCE] Refactor SiteIndexBuilder.build() (#3789)
- [MAINTENANCE] Update ref to ge-cla-bot in PR template (#3799)
- [MAINTENANCE] Anonymizer clean up and refactor (#3801)
- [MAINTENANCE] Certify the expectation "expect_table_row_count_to_equal_other_table" for V3 API (#3803)
- [MAINTENANCE] Refactor to enable broader use of event emitting method for usage statistics (#3825)
- [MAINTENANCE] Clean up temp file after CI/CD run (#3823)
- [MAINTENANCE] Raising exceptions for misconfigured datasources in cloud mode (#3866)
- [FEATURE] Feature/render validation metadata (#3397) (thanks @vshind1)
- [FEATURE] Added expectation expect_column_values_to_not_contain_special_characters() (#2849, #3771) (thanks @jaibirsingh)
- [FEATURE] Like and regex-based expectations in Athena dialect (#3762) (thanks @josges)
- [FEATURE] Rename
deep_filter_properties_dict()
todeep_filter_properties_iterable()
- [FEATURE] Extract validation result failures (#3552) (thanks @BenGale93)
- [BUGFIX] Allow now() eval parameter to be used by itself (#3719)
- [BUGFIX] Fixing broken logo for legacy RTD docs (#3769)
- [BUGFIX] Adds version-handling to sqlalchemy make_url imports (#3768)
- [BUGFIX] Integration test to avoid regression of simple PandasExecutionEngine workflow (#3770)
- [BUGFIX] Fix copying of CheckpointConfig for substitution and printing purposes (#3759)
- [BUGFIX] Fix evaluation parameter usage with Query Store (#3763)
- [BUGFIX] Feature/fix row condition quotes (#3676) (thanks @benoitLebreton-perso)
- [BUGFIX] Fix incorrect filling out of anonymized event payload (#3780)
- [BUGFIX] Don't reset_index for conditional expectations (#3667) (thanks @abekfenn)
- [DOCS] Update expectations gallery link in V3 notebook documentation (#3747)
- [DOCS] Correct V3 documentation link in V2 notebooks to point to V2 documentation (#3750)
- [DOCS] How to pass an in-memory DataFrame to a Checkpoint (#3756)
- [MAINTENANCE] Fix typo in Getting Started Guide (#3749)
- [MAINTENANCE] Add proper docstring and type hints to Validator (#3767)
- [MAINTENANCE] Clean up duplicate logging statements about optional
black
dep (#3778)
- [FEATURE] Add new result_format to include unexpected_row_list (#3346)
- [FEATURE] Implement "deep_filter_properties_dict()" method (#3703)
- [FEATURE] Create Constants for GETTING_STARTED Entities (e.g., datasource_name, expectation_suite_name, etc.) (#3712)
- [FEATURE] Add usage statistics event for DataContext.get_batch_list() method (#3708)
- [FEATURE] Add data_context.run_checkpoint event to usage statistics (#3721)
- [FEATURE] Add event_duration to usage statistics events (#3729)
- [FEATURE] InferredAssetSqlDataConnector's introspection can list external tables in Redshift Spectrum (#3646)
- [BUGFIX] Using a RuntimeBatchRequest in a Checkpoint with a top-level batch_request instead of validations (#3680)
- [BUGFIX] Using a RuntimeBatchRequest in a Checkpoint at runtime with Checkpoint.run() (#3713)
- [BUGFIX] Using a RuntimeBatchRequest in a Checkpoint at runtime with context.run_checkpoint() (#3718)
- [BUGFIX] Use SQLAlchemy make_url helper where applicable when parsing URLs (#3722)
- [BUGFIX] Adds check for quantile_ranges to be ordered or unbounded pairs (#3724)
- [BUGFIX] Updates MST renderer to return JSON-parseable boolean (#3728)
- [BUGFIX] Removes sqlite suppression for expect_column_quantile_values_to_be_between test definitions (#3735)
- [BUGFIX] Handle contradictory configurations in checkpoint.yml, checkpoint.run(), and context.run_checkpoint() (#3723)
- [BUGFIX] fixed a bug where expectation metadata doesn't appear in edit template for table-level expectations (#3129) (thanks @olechiw)
- [BUGFIX] Added temp_table creation for Teradata in SqlAlchemyBatchData (#3731) (thanks @imamolp)
- [DOCS] Add Databricks video walkthrough link (#3702, #3704)
- [DOCS] Update the link to configure a MetricStore (#3711, #3714) (thanks @txblackbird)
- [DOCS] Updated code example to remove deprecated "File" function (#3632) (thanks @daccorti)
- [DOCS] Delete how_to_add_a_validation_operator.md as OBE. (#3734)
- [DOCS] Update broken link in FOOTER.md to point to V3 documentation (#3745)
- [MAINTENANCE] Improve type hinting (using Optional type) (#3709)
- [MAINTENANCE] Standardize names for assets that are used in Getting Started Guide (#3706)
- [MAINTENANCE] Clean up remaining improper usage of Optional type annotation (#3710)
- [MAINTENANCE] Refinement of Getting Started Guide script (#3715)
- [MAINTENANCE] cloud-410 - Support for Column Descriptions (#3707)
- [MAINTENANCE] Types Clean Up in Checkpoint, Batch, and DataContext Classes (#3737)
- [MAINTENANCE] Remove DeprecationWarning for validator.remove_expectation (#3744)
- [FEATURE] Enable support for Teradata SQLAlchemy dialect (#3496) (thanks @imamolp)
- [FEATURE] Dremio connector added (SQLalchemy) (#3624) (thanks @chufe-dremio)
- [FEATURE] Adds expect_column_values_to_be_string_integers_increasing (#3642)
- [FEATURE] Enable "column.quantile_values" and "expect_column_quantile_values_to_be_between" for SQLite; add/enable new tests (#3695)
- [BUGFIX] Allow glob_directive for DBFS Data Connectors (#3673)
- [BUGFIX] Update black version in pre-commit config (#3674)
- [BUGFIX] Make sure to add "mostly_pct" value if "mostly" kwarg present (#3661)
- [BUGFIX] Fix BatchRequest.to_json_dict() to not overwrite original fields; also type usage cleanup in CLI tests (#3683)
- [BUGFIX] Fix pyfakefs boto / GCS incompatibility (#3694)
- [BUGFIX] Update prefix attr assignment in cloud-based DataConnector constructors (#3668)
- [BUGFIX] Update 'list_keys' signature for all cloud-based tuple store child classes (#3669)
- [BUGFIX] evaluation parameters from different expectation suites dependencies (#3684) (thanks @OmriBromberg)
- [DOCS] Databricks deployment pattern documentation (#3682)
- [DOCS] Remove how_to_instantiate_a_data_context_on_databricks_spark_cluster (#3687)
- [DOCS] Updates to Databricks doc based on friction logging (#3696)
- [MAINTENANCE] Fix checkpoint anonymization and make BatchRequest.to_json_dict() more robust (#3675)
- [MAINTENANCE] Update kl_divergence domain_type (#3681)
- [MAINTENANCE] update filter_properties_dict to use set for inclusions and exclusions (instead of list) (#3698)
- [MAINTENANCE] Adds CITATION.cff (#3697)
- [FEATURE] DBFS Data connectors (#3659)
- [BUGFIX] Fix "null" appearing in notebooks due to incorrect ExpectationConfigurationSchema serialization (#3638)
- [BUGFIX] Ensure that result_format from saved expectation suite json file takes effect (#3634)
- [BUGFIX] Allowing user specified run_id to appear in WarningAndFailureExpectationSuitesValidationOperator validation result (#3386) (thanks @wniroshan)
- [BUGFIX] Update black dependency to ensure passing Azure builds on Python 3.9 (#3664)
- [BUGFIX] fix Issue #3405 - gcs client init in pandas engine (#3408) (thanks @dz-1)
- [BUGFIX] Recursion error when passing RuntimeBatchRequest with query into Checkpoint using validations (#3654)
- [MAINTENANCE] Cloud 388/supported expectations query (#3635)
- [MAINTENANCE] Proper separation of concerns between specific File Path Data Connectors and corresponding ExecutionEngine objects (#3643)
- [MAINTENANCE] Enable Docusaurus tests for S3 (#3645)
- [MAINTENANCE] Formalize Exception Handling Between DataConnector and ExecutionEngine Implementations, and Update DataConnector Configuration Usage in Tests (#3644)
- [MAINTENANCE] Adds util for handling SADeprecation warning (#3651)
- [FEATURE] Support median calculation in AWS Athena (#3596) (thanks @persiyanov)
- [BUGFIX] Be able to use spark execution engine with spark reuse flag (#3541) (thanks @fep2)
- [DOCS] punctuation how_to_contribute_a_new_expectation_to_great_expectations.md (#3484) (thanks @plain-jane-gray)
- [DOCS] Update next_steps.md (#3483) (thanks @plain-jane-gray)
- [DOCS] Update how_to_configure_a_validation_result_store_in_gcs.md (#3482) (thanks @plain-jane-gray)
- [DOCS] Choosing and configuring DataConnectors (#3533)
- [DOCS] Remove --no-spark flag from docs tests (#3625)
- [DOCS] DevRel - docs fixes (#3498)
- [DOCS] Adding a period (#3627) (thanks @plain-jane-gray)
- [DOCS] Remove comments that describe Snowflake parameters as optional (#3639)
- [MAINTENANCE] Update CODEOWNERS (#3604)
- [MAINTENANCE] Fix logo (#3598)
- [MAINTENANCE] Add Expectations to docs navbar (#3597)
- [MAINTENANCE] Remove unused fixtures (#3218)
- [MAINTENANCE] Remove unnecessary comment (#3608)
- [MAINTENANCE] Superconductive Warnings hackathon (#3612)
- [MAINTENANCE] Bring Core Skills Doc for Creating Batch Under Test (#3629)
- [MAINTENANCE] Refactor and Clean Up Expectations and Metrics Parts of the Codebase (better encapsulation, improved type hints) (#3633)
- [FEATURE] Retrieve data context config through Cloud API endpoint #3586
- [FEATURE] Update Batch IDs to match name change in paths included in batch_request #3587
- [FEATURE] V2-to-V3 Upgrade/Migration #3592
- [FEATURE] table and graph atomic renderers #3595
- [FEATURE] V2-to-V3 Upgrade/Migration (Sidebar.js update) #3603
- [DOCS] Fixing broken links and linking to Expectation Gallery #3591
- [MAINTENANCE] Get TZLocal back to its original version control. #3585
- [MAINTENANCE] Add tests for datetime evaluation parameters #3601
- [MAINTENANCE] Removed warning for pandas option display.max_colwidth #3606
- [FEATURE] Migration of Expectations to Atomic Prescriptive Renderers (#3530, #3537)
- [FEATURE] Cloud: Editing Expectation Suites programmatically (#3564)
- [BUGFIX] Fix deprecation warning for importing from collections (#3546) (thanks @shpolina)
- [BUGFIX] SQLAlchemy version 1.3.24 compatibility in map metric provider (#3507) (thanks @shpolina)
- [DOCS] Clarify how to configure optional Snowflake parameters in CLI datasource new notebook (#3543)
- [DOCS] Added breaks to code snippets, reordered guidance (#3514)
- [DOCS] typo in documentation (#3542) (thanks @DanielEdu)
- [DOCS] Update how_to_configure_a_new_data_context_with_the_cli.md (#3556) (thanks @plain-jane-gray)
- [DOCS] Improved installation instructions, included in-line installation instructions to getting started (#3509)
- [DOCS] Update contributing_style.md (#3521) (thanks @plain-jane-gray)
- [DOCS] Update contributing_test.md (#3519) (thanks @plain-jane-gray)
- [DOCS] Revamp style guides (#3554)
- [DOCS] Update contributing.md (#3523, #3524) (thanks @plain-jane-gray)
- [DOCS] Simplify getting started (#3555)
- [DOCS] How to introspect and partition an SQL database (#3465)
- [DOCS] Update contributing_checklist.md (#3518) (thanks @plain-jane-gray)
- [DOCS] Removed duplicate prereq, how_to_instantiate_a_data_context_without_a_yml_file.md (#3481) (thanks @plain-jane-gray)
- [DOCS] fix link to expectation glossary (#3558) (thanks @sephiartlist)
- [DOCS] Minor Friction (#3574)
- [MAINTENANCE] Make CLI Check-Config and CLI More Robust (#3562)
- [MAINTENANCE] tzlocal version fix (#3565)
- [FEATURE] Atomic Renderer: Initial framework and Prescriptive renderers (#3529)
- [FEATURE] Atomic Renderer: Diagnostic renderers (#3534)
- [BUGFIX] runtime_parameters: {batch_data: Spark DF} serialization (#3502)
- [BUGFIX] Custom query in RuntimeBatchRequest for expectations using table.row_count metric (#3508)
- [BUGFIX] Transpose \n and , in notebook (#3463) (thanks @mccalluc)
- [BUGFIX] Fix contributor link (#3462) (thanks @mccalluc)
- [DOCS] How to introspect and partition a files based data store (#3464)
- [DOCS] fixed duplication of text in code example (#3503)
- [DOCS] Make content better reflect the document organization. (#3510)
- [DOCS] Correcting typos and improving the language. (#3513)
- [DOCS] Better Sections Numbering in Documentation (#3515)
- [DOCS] Improved wording (#3516)
- [DOCS] Improved title wording for section heading (#3517)
- [DOCS] Improve Readability of Documentation Content (#3536)
- [MAINTENANCE] Content and test script update (#3532)
- [MAINTENANCE] Provide Deprecation Notice for the "parse_strings_as_datetimes" Expectation Parameter in V3 (#3539)
- [FEATURE] Implement CompoundColumnsUnique metric for SqlAlchemyExecutionEngine (#3477)
- [FEATURE] add get_available_data_asset_names_and_types (#3476)
- [FEATURE] add s3_put_options to TupleS3StoreBackend (#3470) (Thanks @kj-9)
- [BUGFIX] Fix TupleS3StoreBackend remove_key bug (#3489)
- [DOCS] Adding Flyte Deployment pattern to docs (#3383)
- [DOCS] g_e docs branding updates (#3471)
- [MAINTENANCE] Add type-hints; add utility method for creating temporary DB tables; clean up imports; improve code readability; and add a directory to pre-commit (#3475)
- [MAINTENANCE] Clean up for a better code readability. (#3493)
- [MAINTENANCE] Enable SQL for the "expect_compound_columns_to_be_unique" expectation. (#3488)
- [MAINTENANCE] Fix some typos (#3474) (Thanks @mohamadmansourX)
- [MAINTENANCE] Support SQLAlchemy version 1.3.24 for compatibility with Airflow (Airflow does not currently support later versions of SQLAlchemy). (#3499)
- [MAINTENANCE] Update contributing_checklist.md (#3478) (Thanks @plain-jane-gray)
- [MAINTENANCE] Update how_to_configure_a_validation_result_store_in_gcs.md (#3480) (Thanks @plain-jane-gray)
- [MAINTENANCE] update implemented_expectations (#3492)
- [FEATURE] GREAT-3439 extended SlackNotificationsAction for slack app tokens (#3440) (Thanks @psheets)
- [FEATURE] Implement Integration Test for "Simple SQL Datasource" with Partitioning, Splitting, and Sampling (#3454)
- [FEATURE] Implement Integration Test for File Path Data Connectors with Partitioning, Splitting, and Sampling (#3452)
- [BUGFIX] Fix Incorrect Implementation of the "_sample_using_random" Sampling Method in SQLAlchemyExecutionEngine (#3449)
- [BUGFIX] Handle RuntimeBatchRequest passed to Checkpoint programatically (without yml) (#3448)
- [DOCS] Fix typo in command to create new checkpoint (#3434) (Thanks @joeltone)
- [DOCS] How to validate data by running a Checkpoint (#3436)
- [ENHANCEMENT] cloud-199 - Update Expectation and ExpectationSuite classes for GE Cloud (#3453)
- [MAINTENANCE] Does not test numpy.float128 when it doesn't exist (#3460)
- [MAINTENANCE] Remove Unnecessary SQL OR Condition (#3469)
- [MAINTENANCE] Remove validation playground notebooks (#3467)
- [MAINTENANCE] clean up type hints, API usage, imports, and coding style (#3444)
- [MAINTENANCE] comments (#3457)
- [FEATURE] Create ExpectationValidationGraph class to Maintain Relationship Between Expectation and Metrics and Use it to Associate Exceptions to Expectations (#3433)
- [BUGFIX] Addresses issue #2993 (#3054) by using configuration when it is available instead of discovering keys (listing keys) in existing sources. (#3377)
- [BUGFIX] Fix Data asset name rendering (#3431) (Thanks @shpolina)
- [DOCS] minor fix to syntax highlighting in how_to_contribute_a_new_expectation… (#3413) (Thanks @edjoesu)
- [DOCS] Fix broken links in how_to_create_a_new_expectation_suite_using_rule_based_profile… (#3410) (Thanks @edjoesu)
- [ENHANCEMENT] update list_expectation_suite_names and ExpectationSuiteValidationResult payload (#3419)
- [MAINTENANCE] Clean up Type Hints, JSON-Serialization, ID Generation and Logging in Objects in batch.py Module and its Usage (#3422)
- [MAINTENANCE] Fix Granularity of Exception Handling in ExecutionEngine.resolve_metrics() and Clean Up Type Hints (#3423)
- [MAINTENANCE] Fix broken links in how_to_create_a_new_expectation_suite_using_rule_based_profiler (#3441)
- [MAINTENANCE] Fix issue where BatchRequest object in configuration could cause Checkpoint to fail (#3438)
- [MAINTENANCE] Insure consistency between implementation of overriding Python hash() and internal ID property value (#3432)
- [MAINTENANCE] Performance improvement refactor for Spark unexpected values (#3368)
- [MAINTENANCE] Refactor MetricConfiguration out of validation_graph.py to Avoid Future Circular Dependencies in Python (#3425)
- [MAINTENANCE] Use ExceptionInfo to encapsulate common expectation validation result error information. (#3427)
- [FEATURE] Configurable multi-threaded checkpoint speedup (#3362) (Thanks @jdimatteo)
- [BUGFIX] Insure that the "result_format" Expectation Argument is Processed Properly (#3364)
- [BUGFIX] fix error getting validation result from DataContext (#3359) (Thanks @zachzIAM)
- [BUGFIX] fixed typo and added CLA links (#3347)
- [DOCS] Azure Data Connector Documentation for Pandas and Spark. (#3378)
- [DOCS] Connecting to GCS using Spark (#3375)
- [DOCS] Docusaurus - Deploying Great Expectations in a hosted environment without file system or CLI (#3361)
- [DOCS] How to get a batch from configured datasource (#3382)
- [MAINTENANCE] Add Flyte to README (#3387) (Thanks @samhita-alla)
- [MAINTENANCE] Adds expect_table_columns_to_match_set (#3329) (Thanks @viniciusdsmello)
- [MAINTENANCE] Bugfix/skip substitute config variables in ge cloud mode (#3393)
- [MAINTENANCE] Clean Up ValidationGraph API Usage, Improve Exception Handling for Metrics, Clean Up Type Hints (#3399)
- [MAINTENANCE] Clean up ValidationGraph API and add Type Hints (#3392)
- [MAINTENANCE] Enhancement/update _set methods with kwargs (#3391) (Thanks @roblim)
- [MAINTENANCE] Fix incorrect ToC section name (#3395)
- [MAINTENANCE] Insure Correct Processing of the catch_exception Flag in Metrics Resolution (#3360)
- [MAINTENANCE] exempt batch_data from a deep_copy operation on RuntimeBatchRequest (#3388)
- [MAINTENANCE] [WIP] Enhancement/cloud 169/update checkpoint.run for ge cloud (#3381)
- [FEATURE] Add optional ge_cloud_mode flag to DataContext to enable use with Great Expectations Cloud.
- [FEATURE] Rendered Data Doc JSONs can be uploaded and retrieved from GE Cloud
- [FEATURE] Implement InferredAssetAzureDataConnector with Support for Pandas and Spark Execution Engines (#3372)
- [FEATURE] Spark connecting to Google Cloud Storage (#3365)
- [FEATURE] SparkDFExecutionEngine can load data accessed by ConfiguredAssetAzureDataConnector (integration tests are included). (#3345)
- [FEATURE] [MER-293] GE Cloud Mode for DataContext (#3262) (Thanks @roblim)
- [BUGFIX] Allow for RuntimeDataConnector to accept custom query while suppressing temp table creation (#3335) (Thanks @NathanFarmer)
- [BUGFIX] Fix issue where multiple validators reused the same execution engine, causing a conflict in active batch (GE-3168) (#3222) (Thanks @jcampbell)
- [BUGFIX] Run batch_request dictionary through util function convert_to_json_serializable (#3349) (Thanks @NathanFarmer)
- [BUGFIX] added casting of numeric value to fix redshift issue #3293 (#3338) (Thanks @sariabod)
- [DOCS] Docusaurus - How to connect to an MSSQL database (#3353) (Thanks @NathanFarmer)
- [DOCS] GREAT-195 Docs remove all stubs and links to them (#3363)
- [MAINTENANCE] Update azure-pipelines-docs-integration.yml for Azure Pipelines
- [MAINTENANCE] Update implemented_expectations.md (#3351) (Thanks @spencerhardwick)
- [MAINTENANCE] Updating to reflect current Expectation dev state (#3348) (Thanks @spencerhardwick)
- [MAINTENANCE] docs: Clean up Docusaurus refs (#3371)
- [FEATURE] Add Performance Benchmarks Using BigQuery. (Thanks @jdimatteo)
- [WIP] [FEATURE] add backend args to run_diagnostics (#3257) (Thanks @edjoesu)
- [BUGFIX] Addresses Issue 2937. (#3236) (Thanks @BenGale93)
- [BUGFIX] SQL dialect doesn't register for BigQuery for V2 (#3324)
- [DOCS] "How to connect to data on GCS using Pandas" (#3311)
- [MAINTENANCE] Add CODEOWNERS with a single check for sidebars.js (#3332)
- [MAINTENANCE] Fix incorrect DataConnector usage of _get_full_file_path() API method. (#3336)
- [MAINTENANCE] Make Pandas against S3 and GCS integration tests more robust by asserting on number of batches returned and row counts (#3341)
- [MAINTENANCE] Make integration tests of Pandas against Azure more robust. (#3339)
- [MAINTENANCE] Prepare AzureUrl to handle WASBS format (for Spark) (#3340)
- [MAINTENANCE] Renaming default_batch_identifier in examples #3334
- [MAINTENANCE] Tests for RuntimeDataConnector at DataContext-level (#3304)
- [MAINTENANCE] Tests for RuntimeDataConnector at DataContext-level (Spark and Pandas) (#3325)
- [MAINTENANCE] Tests for RuntimeDataConnector at Datasource-level (Spark and Pandas) (#3318)
- [MAINTENANCE] Various doc patches (#3326)
- [MAINTENANCE] clean up imports and method signatures (#3337)
- [FEATURE] Enable
GCS DataConnector
integration withPandasExecutionEngine
(#3264) - [FEATURE] Enable column_pair expectations and tests for Spark (#3294)
- [FEATURE] Implement
InferredAssetGCSDataConnector
(#3284) - [FEATURE]/CHANGE run time format (#3272) (Thanks @serialbandicoot)
- [DOCS] Fix misc errors in "How to create renderers for Custom Expectations" (#3315)
- [DOCS] GDOC-217 remove stub links (#3314)
- [DOCS] Remove misc TODOs to tidy up docs (#3313)
- [DOCS] Standardize capitalization of various technologies in
docs
(#3312) - [DOCS] Fix broken link to Contributor docs (#3295) (Thanks @discdiver)
- [MAINTENANCE] Additional tests for RuntimeDataConnector at Datasource-level (query) (#3288)
- [MAINTENANCE] Update GCSStoreBackend + tests (#2630) (Thanks @hmandsager)
- [MAINTENANCE] Write integration/E2E tests for
ConfiguredAssetAzureDataConnector
(#3204) - [MAINTENANCE] Write integration/E2E tests for both
GCSDataConnectors
(#3301)
- [FEATURE] Implement Spark Decorators and Helpers; Demonstrate on MulticolumnSumEqual Metric (#3289)
- [FEATURE] V3 implement expect_column_pair_values_to_be_in_set for SQL Alchemy execution engine (#3281)
- [FEATURE] Implement
ConfiguredAssetGCSDataConnector
(#3247) - [BUGFIX] Fix import issues around cloud providers (GCS/Azure/S3) (#3292)
- [MAINTENANCE] Add force_reuse_spark_context to DatasourceConfigSchema (#3126) (thanks @gipaetusb and @mbakunze)
- [FEATURE] Implementation of the Metric "select_column_values.unique.within_record" for SQLAlchemyExecutionEngine (#3279)
- [FEATURE] V3 implement ColumnPairValuesInSet for SQL Alchemy execution engine (#3278)
- [FEATURE] Edtf with support levels (#2594) (thanks @mielvds)
- [FEATURE] V3 implement expect_column_pair_values_to_be_equal for SqlAlchemyExecutionEngine (#3267)
- [FEATURE] add expectation for discrete column entropy (#3049) (thanks @edjoesu)
- [FEATURE] Add SQLAlchemy Provider for the the column_pair_values.a_greater_than_b Metric (#3268)
- [FEATURE] Expectations tests for BigQuery backend (#3219) (Thanks @jdimatteo)
- [FEATURE] Add schema validation for different GCS auth methods (#3258)
- [FEATURE] V3 - Implement column_pair helpers/providers for SqlAlchemyExecutionEngine (#3256)
- [FEATURE] V3 implement expect_column_pair_values_to_be_equal expectation for PandasExecutionEngine (#3252)
- [FEATURE] GCS DataConnector schema validation (#3253)
- [FEATURE] Implementation of the "expect_select_column_values_to_be_unique_within_record" Expectation (#3251)
- [FEATURE] Implement the SelectColumnValuesUniqueWithinRecord metric (for PandasExecutionEngine) (#3250)
- [FEATURE] V3 - Implement ColumnPairValuesEqual for PandasExecutionEngine (#3243)
- [FEATURE] Set foundation for GCS DataConnectors (#3220)
- [FEATURE] Implement "expect_column_pair_values_to_be_in_set" expectation (support for PandasExecutionEngine) (#3242)
- [BUGFIX] Fix deprecation warning for importing from collections (#3228) (thanks @ismaildawoodjee)
- [DOCS] Document BigQuery test dataset configuration (#3273) (Thanks @jdimatteo)
- [DOCS] Syntax and Link (#3266)
- [DOCS] API Links and Supporting Docs (#3265)
- [DOCS] redir and search (#3249)
- [MAINTENANCE] Update azure-pipelines-docs-integration.yml to include env vars for Azure docs integration tests
- [MAINTENANCE] Allow Wrong ignore_row_if Directive from V2 with Deprecation Warning (#3274)
- [MAINTENANCE] Refactor test structure for "Connecting to your data" cloud provider integration tests (#3277)
- [MAINTENANCE] Make test method names consistent for Metrics tests (#3254)
- [MAINTENANCE] Allow
PandasExecutionEngine
to acceptAzure DataConnectors
(#3214) - [MAINTENANCE] Standardize Arguments to MetricConfiguration Constructor; Use {} instead of dict(). (#3246)
- [FEATURE] Implement ColumnPairValuesInSet metric for PandasExecutionEngine
- [BUGFIX] Wrap optional azure imports in data_connector setup
- [FEATURE] Accept row_condition (with condition_parser) and ignore_row_if parameters for expect_multicolumn_sum_to_equal (#3193)
- [FEATURE] ConfiguredAssetDataConnector for Azure Blob Storage (#3141)
- [FEATURE] Replace MetricFunctionTypes.IDENTITY domain type with convenience method get_domain_records() for SparkDFExecutionEngine (#3226)
- [FEATURE] Replace MetricFunctionTypes.IDENTITY domain type with convenience method get_domain_records() for SqlAlchemyExecutionEngine (#3215)
- [FEATURE] Replace MetricFunctionTypes.IDENTITY domain type with convenience method get_full_access_compute_domain() for PandasExecutionEngine (#3210)
- [FEATURE] Set foundation for Azure-related DataConnectors (#3188)
- [FEATURE] Update ExpectCompoundColumnsToBeUnique for V3 API (#3161)
- [BUGFIX] Fix incorrect schema validation for Azure data connectors (#3200)
- [BUGFIX] Fix incorrect usage of "all()" in the comparison of validation results when executing an Expectation (#3178)
- [BUGFIX] Fixes an error with expect_column_values_to_be_dateutil_parseable (#3190)
- [BUGFIX] Improve parsing of .ge_store_backend_id (#2952)
- [BUGFIX] Remove fixture parameterization for Cloud DBs (Snowflake and BigQuery) (#3182)
- [BUGFIX] Restore support for V2 API style custom expectation rendering (#3179) (Thanks @jdimatteo)
- [DOCS] Add
conda
as installation option in README (#3196) (Thanks @rpanai) - [DOCS] Standardize capitalization of "Python" in "Connecting to your data" section of new docs (#3209)
- [DOCS] Standardize capitalization of Spark in docs (#3198)
- [DOCS] Update BigQuery docs to clarify the use of temp tables (#3184)
- [DOCS] Create _redirects (#3192)
- [ENHANCEMENT] RuntimeDataConnector messaging is made more clear for
test_yaml_config()
(#3206) - [MAINTENANCE] Add
credentials
YAML key support forDataConnectors
(#3173) - [MAINTENANCE] Fix minor typo in S3 DataConnectors (#3194)
- [MAINTENANCE] Fix typos in argument names and types (#3207)
- [MAINTENANCE] Update changelog. (#3189)
- [MAINTENANCE] Update documentation. (#3203)
- [MAINTENANCE] Update validate_your_data.md (#3185)
- [MAINTENANCE] update tests across execution engines and clean up coding patterns (#3223)
- [FEATURE] Enable BigQuery tests for Azure CI/CD (#3155)
- [FEATURE] Implement MulticolumnMapExpectation class (#3134)
- [FEATURE] Implement the MulticolumnSumEqual Metric for PandasExecutionEngine (#3130)
- [FEATURE] Support row_condition and ignore_row_if Directives Combined for PandasExecutionEngine (#3150)
- [FEATURE] Update ExpectMulticolumnSumToEqual for V3 API (#3136)
- [FEATURE] add python3.9 to python versions (#3143) (Thanks @dswalter)
- [FEATURE]/MER-16/MER-75/ADD_ROUTE_FOR_VALIDATION_RESULT (#3090) (Thanks @rreinoldsc)
- [BUGFIX] Enable
--v3-api suite edit
to proceed without selecting DataConnectors (#3165) - [BUGFIX] Fix error when
RuntimeBatchRequest
is passed toSimpleCheckpoint
withRuntimeDataConnector
(#3152) - [BUGFIX] allow reader_options in the CLI so can read
.csv.gz
files (#2695) (Thanks @luke321321) - [DOCS] Apply Docusaurus tabs to relevant pages in new docs
- [DOCS] Capitalize python to Python in docs (#3176)
- [DOCS] Improve Core Concepts - Expectation Concepts (#2831)
- [MAINTENANCE] Error messages must be friendly. (#3171)
- [MAINTENANCE] Implement the "compound_columns_unique" metric for PandasExecutionEngine (with a unit test). (#3159)
- [MAINTENANCE] Improve Coding Practices in "great_expectations/expectations/expectation.py" (#3151)
- [MAINTENANCE] Update test_script_runner.py (#3177)
- [FEATURE] Pass on meta-data from expectation json to validation result json (#2881) (Thanks @sushrut9898)
- [FEATURE] Add sqlalchemy engine support for
column.most_common_value
metric (#3020) (Thanks @shpolina) - [BUGFIX] Added newline to CLI message for consistent formatting (#3127) (Thanks @ismaildawoodjee)
- [BUGFIX] fix pip install snowflake build error with Python 3.9 (#3119) (Thanks @jdimatteo)
- [BUGFIX] Populate (data) asset name in data docs for RuntimeDataConnector (#3105) (Thanks @ceshine)
- [DOCS] Correct path to docs_rtd/changelog.rst (#3120) (Thanks @jdimatteo)
- [DOCS] Fix broken links in "How to write a 'How to Guide'" (#3112)
- [DOCS] Port over "How to add comments to Expectations and display them in DataDocs" from RTD to Docusaurus (#3078)
- [DOCS] Port over "How to create a Batch of data from an in memory Spark or Pandas DF" from RTD to Docusaurus (#3099)
- [DOCS] Update CLI codeblocks in create_your_first_expectations.md (#3106) (Thanks @ories)
- [MAINTENANCE] correct typo in docstring (#3117)
- [MAINTENANCE] DOCS/GDOC-130/Add Changelog (#3121)
- [MAINTENANCE] fix docstring for expectation "expect_multicolumn_sum_to_equal" (previous version was not precise) (#3110)
- [MAINTENANCE] Fix typos in docstrings in map_metric_provider partials (#3111)
- [MAINTENANCE] Make sure that all imports use column_aggregate_metric_provider (not column_aggregate_metric). (#3128)
- [MAINTENANCE] Rename column_aggregate_metric.py into column_aggregate_metric_provider.py for better code readability. (#3123)
- [MAINTENANCE] rename ColumnMetricProvider to ColumnAggregateMetricProvider (with DeprecationWarning) (#3100)
- [MAINTENANCE] rename map_metric.py to map_metric_provider.py (with DeprecationWarning) for a better code readability/interpretability (#3103)
- [MAINTENANCE] rename table_metric.py to table_metric_provider.py with a deprecation notice (#3118)
- [MAINTENANCE] Update CODE_OF_CONDUCT.md (#3066)
- [MAINTENANCE] Upgrade to modern Python syntax (#3068) (Thanks @cclauss)
- [FEATURE] Script to automate proper triggering of Docs Azure pipeline (#3003)
- [BUGFIX] Fix an undefined name that could lead to a NameError (#3063) (Thanks @cclauss)
- [BUGFIX] fix incorrect pandas top rows usage (#3091)
- [BUGFIX] Fix parens in Expectation metric validation method that always returned True assertation (#3086) (Thanks @morland96)
- [BUGFIX] Fix run_diagnostics for contrib expectations (#3096)
- [BUGFIX] Fix typos discovered by codespell (#3064) (Thanks cclauss)
- [BUGFIX] Wrap get_view_names in try clause for passing the NotImplemented error (#2976) (Thanks @kj-9)
- [DOCS] Ensuring consistent style of directories, files, and related references in docs (#3053)
- [DOCS] Fix broken link to example DAG (#3061) (Thanks fritz-astronomer)
- [DOCS] GDOC-198 cleanup TOC (#3088)
- [DOCS] Migrating pages under guides/miscellaneous (#3094) (Thanks @spbail)
- [DOCS] Port over “How to configure a new Checkpoint using test_yaml_config” from RTD to Docusaurus
- [DOCS] Port over “How to configure an Expectation store in GCS” from RTD to Docusaurus (#3071)
- [DOCS] Port over “How to create renderers for custom Expectations” from RTD to Docusaurus
- [DOCS] Port over “How to run a Checkpoint in Airflow” from RTD to Docusaurus (#3074)
- [DOCS] Update how-to-create-and-edit-expectations-in-bulk.md (#3073)
- [MAINTENANCE] Adding a comment explaining the IDENTITY metric domain type. (#3057)
- [MAINTENANCE] Change domain key value from “column” to “column_list” in ExecutionEngine implementations (#3059)
- [MAINTENANCE] clean up metric errors (#3085)
- [MAINTENANCE] Correct the typo in the naming of the IDENTIFICATION semantic domain type name. (#3058)
- [MAINTENANCE] disable snowflake tests temporarily (#3093)
- [MAINTENANCE] [DOCS] Port over “How to host and share Data Docs on GCS” from RTD to Docusaurus (#3070)
- [MAINTENANCE] Enable repr for MetricConfiguration to assist with troubleshooting. (#3075)
- [MAINTENANCE] Expand test of a column map metric to underscore functionality. (#3072)
- [MAINTENANCE] Expectation anonymizer supports v3 expectation registry (#3092)
- [MAINTENANCE] Fix -- check for column key existence in accessor_domain_kwargsn for condition map partials. (#3082)
- [MAINTENANCE] Missing import of SparkDFExecutionEngine was added. (#3062)
Older changelist can be found at https://github.com/great-expectations/great_expectations/blob/develop/docs_rtd/changelog.rst