Skip to content

Commit

Permalink
Merge branch 'feature/cli_refactor_rebase' into feature/data_context_…
Browse files Browse the repository at this point in the history
…namespace

Key conflicts came from need to harmonize profiling get_batch and get_expectations to new namespaced API.

Conflicts:
	docs/source/profiling.rst
	great_expectations/data_asset/data_asset.py
	great_expectations/util.py
	tests/conftest.py
	tests/test_autoinspect.py
  • Loading branch information
jcampbell committed Jun 17, 2019
2 parents a17d2a0 + f799675 commit c9fbefc
Show file tree
Hide file tree
Showing 34 changed files with 2,432 additions and 504 deletions.
8 changes: 0 additions & 8 deletions docs/source/dataset_module.rst
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,3 @@ great_expectations.dataset.util
:undoc-members:
:show-inheritance:


great_expectations.dataset.autoinspect
--------------------------------------

.. automodule:: great_expectations.dataset.autoinspect
:members:
:undoc-members:
:show-inheritance:
3 changes: 2 additions & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Advanced

standard_arguments
result_format
autoinspection
profiling
evaluation_parameters
custom_expectations
conventions
Expand All @@ -40,6 +40,7 @@ Module Docs
data_asset_module
dataset_module
data_context_module
profile_module

Indices and tables
------------------
Expand Down
12 changes: 12 additions & 0 deletions docs/source/profile.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
.. _profile_module:

Profile Module
==================================

great_expectations.profile
--------------------------------------

.. automodule:: great_expectations.profile
:members:
:undoc-members:
:show-inheritance:
20 changes: 6 additions & 14 deletions docs/source/autoinspection.rst → docs/source/profiling.rst
Original file line number Diff line number Diff line change
@@ -1,29 +1,21 @@
.. _autoinspection:
.. _profiling:

================================================================================
Autoinspection
Profiling
================================================================================

It can be very convenient to have great expectations automatically review a \
dataset and suggest expectations that may be appropriate. Currently, there's \
a very basic, but easily extensible, autoinspection capability available.

Dataset objects have an `autoinspect` method which allows you to provide a \
function that will evaluate a dataset object and add expectations to it. \
By default `autoinspect` will call the autoinspect function \
:func:`columns_exist <great_expectations.dataset.autoinspect.columns_exist>` \
which will add an `expect_column_to_exist` expectation for each column \
currently present on the dataset.

To implement additional autoinspection functions, you simply take a single \
parameter, a Dataset, and evaluate and add expectations to that object.
a very basic, but easily extensible, profiling capability available.

Dataset objects have a `profile` method which allows you to provide a \
profiler class that will evaluate a dataset object and add expectations to it.

.. code-block:: python
>> import great_expectations as ge
>> df = ge.dataset.PandasDataset({"col": [1, 2, 3, 4, 5]})
>> df.autoinspect(ge.dataset.autoinspect.columns_exist)
>> df.profile(ge.profile.ColumnsExistProfiler)
>> df.get_expectation_suite()
{'dataset_name': None,
'meta': {'great_expectations.__version__': '0.4.4__develop'},
Expand Down
12 changes: 9 additions & 3 deletions docs/source/roadmap_changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,16 @@ Planned Features
* Improved variable typing
* Support for non-tabular datasources (e.g. JSON, XML, AVRO)

v.0.6.1__develop
v.0.7.0__develop
________________


* Name clarifications: we've stopped using the overloaded terms "expectations
config" and "config" and instead use "expectation suite" to refer to a
collection (or suite!) of expectations that can be used for validating a
data asset.
* Major refactor of autoinspect. Autoinspect is now built around a module
called "profile" which provides a class-based structure for building
expectation suites. There is no longer a default "autoinspect_func" --
calling autoinspect requires explicitly passing the desired profiler.

v.0.6.1
________________
Expand Down
Loading

0 comments on commit c9fbefc

Please sign in to comment.