Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MAINTENANCE] Rename and move V17 snippets to legacy docs dir #9374

Merged
merged 34 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
8c9640a
wip
joshua-stauffer Feb 2, 2024
e59a9af
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 2, 2024
7e8dcdb
wip
joshua-stauffer Feb 5, 2024
5962c6f
add v17 snippets
joshua-stauffer Feb 5, 2024
0c18642
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 5, 2024
61ef51d
Merge branch 'develop' into m/v1-153/rename_v17_doc_snippets
joshua-stauffer Feb 5, 2024
d5f1c80
manually update snippet name
joshua-stauffer Feb 5, 2024
63b738b
add missing doc
joshua-stauffer Feb 5, 2024
c4c4435
rename current snippet
joshua-stauffer Feb 5, 2024
d1f314b
manually update snippet
joshua-stauffer Feb 5, 2024
d1ddcab
gnarly result format fix
joshua-stauffer Feb 5, 2024
112420c
update remark script
joshua-stauffer Feb 5, 2024
58dfcab
update datasource universal step snippet name
joshua-stauffer Feb 5, 2024
a80c48f
remove unused snippets with duplicated names
joshua-stauffer Feb 5, 2024
c664959
connect to postgres snippet
joshua-stauffer Feb 5, 2024
a89cd1c
update some more
joshua-stauffer Feb 5, 2024
41d1482
update custom expectations snippets
joshua-stauffer Feb 5, 2024
f633e21
update more expectation snippets
joshua-stauffer Feb 5, 2024
55c8911
update v17 result format doc
joshua-stauffer Feb 5, 2024
61545a4
pandas snippets
joshua-stauffer Feb 5, 2024
5126943
Logg all duplicate snippets
tyler-hoffman Feb 5, 2024
977bcdc
Mass find/replace to prefix with 0.17.23
tyler-hoffman Feb 5, 2024
3249350
Revert "Logg all duplicate snippets"
tyler-hoffman Feb 5, 2024
4100397
Fix the missing snippet errors (I think)
tyler-hoffman Feb 6, 2024
416399b
Merge branch 'develop' into m/v1-153/rename_v17_doc_snippets
joshua-stauffer Feb 6, 2024
7398b46
Fix a link
tyler-hoffman Feb 6, 2024
67fd489
Update href in terms tag
tyler-hoffman Feb 6, 2024
d195c55
Update v0.17 sidebar
tyler-hoffman Feb 7, 2024
8cc449f
Last fixes
tyler-hoffman Feb 7, 2024
c14336b
Merge branch 'develop' into m/v1-153/rename_v17_doc_snippets
tyler-hoffman Feb 7, 2024
88d0ebe
fix links with 0.17.23 prefix
joshua-stauffer Feb 7, 2024
bfcaedc
cleanup
joshua-stauffer Feb 7, 2024
856e8e9
Revert "update remark script"
joshua-stauffer Feb 7, 2024
b631a8a
Revert "Revert "update remark script""
joshua-stauffer Feb 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
```python name="import necessary modules and initialize your data context"
```python name="docs/docusaurus/docs/oss/guides/connecting_to_your_data/datasource_configuration/filesystem_components/how_to_configure_a_datasource_universal_steps imports and data context"
```

The `great_expectations` module will give you access to your Data Context, which is the entry point for working with a Great Expectations project.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,14 @@
https://docs.greatexpectations.io/docs/guides/connecting_to_your_data/datasource_configuration/how_to_configure_a_sql_datasource
"""

# The following imports are used as part of verifying that all example snippets are consistent.
# Users may disregard them.

from datasource_configuration_test_utilities import is_subset
from full_datasource_configurations import (
get_partial_config_universal_datasource_config_elements,
)

# The following methods correspond to the section headings in the how-to guide linked in the module docstring.


def section_1_import_necessary_modules_and_initialize_your_data_context():
"""Provides and tests the snippets for section 1 of the Spark, Pandas, and SQL Datasource configuration guides.

Returns:
a Great Expectations DataContext object
"""
# <snippet name="import necessary modules and initialize your data context">
# <snippet name="docs/docusaurus/docs/oss/guides/connecting_to_your_data/datasource_configuration/filesystem_components/how_to_configure_a_datasource_universal_steps imports and data context">
import great_expectations as gx
from great_expectations.core.yaml_handler import YAMLHandler

Expand All @@ -44,51 +34,4 @@ def section_1_import_necessary_modules_and_initialize_your_data_context():
return data_context


def section_2_create_new_datasource_configuration():
"""Provides and tests the snippets for section 2 of the Spark, Pandas, and SQL Datasource configuration guides."""
# <snippet name="create empty datasource_config dictionary">
datasource_config: dict = {}
# </snippet>
is_subset(
datasource_config, get_partial_config_universal_datasource_config_elements()
)


def section_3_name_your_datasource():
"""Provides and tests the snippets for section 3 of the Spark, Pandas, and SQL Datasource configuration guides."""
# <snippet name="datasource_config up to name being populated as my_datasource_name">
datasource_config: dict = {"name": "my_datasource_name"}
# </snippet>

name_snippet: dict = {
# <snippet name="populate name as my_datasource_name">
"name": "my_datasource_name"
# </snippet>
}
assert name_snippet == datasource_config
is_subset(
datasource_config, get_partial_config_universal_datasource_config_elements()
)


def section_4_specify_the_datasource_class_and_module():
"""Provides and tests the snippets for section 4 of the Spark, Pandas, and SQL Datasource configuration guides."""
# <snippet name="datasource_config containing top level elements universal to spark pandas and sql Datasources">
datasource_config: dict = {
"name": "my_datasource_name", # Preferably name it something relevant
"class_name": "Datasource",
"module_name": "great_expectations.datasource",
}
# </snippet>
assert (
datasource_config == get_partial_config_universal_datasource_config_elements()
)
is_subset(
datasource_config, get_partial_config_universal_datasource_config_elements()
)


section_1_import_necessary_modules_and_initialize_your_data_context()
section_2_create_new_datasource_configuration()
section_3_name_your_datasource()
section_4_specify_the_datasource_class_and_module()
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import NoteExplicitConfiguredAssetsMulti from '../filesystem_components/_note_ex

And the full configuration for your Data Source should look like:

```python name="pandas_configured_datasource_single_batch_full_snippet"
```python name="docs/docusaurus/docs/snippets/how_to_configure_a_pandas_datasource.py pandas_configured_datasource_single_batch_full_snippet"
```

<NoteExplicitConfiguredAssetsSingle />
Expand All @@ -34,7 +34,7 @@ And the full configuration for your Data Source should look like:

And the full configuration for your Data Source should look like:

```python name="pandas_configured_datasource_multi_batch_full_snippet"
```python name="docs/docusaurus/docs/snippets/how_to_configure_a_pandas_datasource.py pandas_configured_datasource_multi_batch_full_snippet"
datasource_config = {
"name": "my_datasource_name",
"class_name": "Data Source",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ import ConfigForAssetsInferredMulti from '../filesystem_components/_config_for_a

And the full configuration for your Data Source should look like:

```python name="pandas_inferred_datasource_single_batch_full_snippet"
```python name="docs/docusaurus/docs/snippets/how_to_configure_a_pandas_datasource.py pandas_inferred_datasource_single_batch_full_snippet"
datasource_config = {
"name": "my_datasource_name",
"class_name": "Data Source",
Expand Down Expand Up @@ -48,7 +48,7 @@ datasource_config = {

And the full configuration for your Data Source should look like:

```python name="pandas_inferred_datasource_multi_batch_full_snippet"
```python name="docs/docusaurus/docs/snippets/how_to_configure_a_pandas_datasource.py pandas_inferred_datasource_multi_batch_full_snippet"
```

</TabItem>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ The following code examples use a PostgreSQL connection string. A PostgreSQL con

The following code is an example of a PostgreSQL connection string format:

```python name="tests/integration/docusaurus/connecting_to_your_data/fluent_datasources/how_to_connect_to_postgreql_data.py connection_string"
```python name="docs/docusaurus/docs/oss/guides/connecting_to_your_data/fluent/database/how_to_connect_to_postgresql_data.py connection_string"
```

:::tip Is there a more secure way to store my credentials than plain text in a connection string?
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
context = gx.get_context()

# Python
# <snippet name="tests/integration/docusaurus/connecting_to_your_data/fluent_datasources/how_to_connect_to_postgreql_data.py connection_string">
# <snippet name="docs/docusaurus/docs/oss/guides/connecting_to_your_data/fluent/database/how_to_connect_to_postgresql_data.py connection_string">
my_connection_string = (
"postgresql+psycopg2://<USERNAME>:<PASSWORD>@<HOST>:<PORT>/<DATABASE>"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ def validate_configuration(
except AssertionError as e:
raise InvalidExpectationConfigurationError(str(e))

# <snippet name="expect_queried_column_value_frequency_to_meet_threshold.py _validate function">
# <snippet name="expect_queried_column_value_frequency_to_meet_threshold.py _validate function signature">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py _validate function">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py _validate function signature">
def _validate(
self,
metrics: dict,
Expand Down Expand Up @@ -111,7 +111,7 @@ def _validate(
}
# </snippet>

# <snippet name="expect_queried_column_value_frequency_to_meet_threshold.py examples">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py examples">
examples = [
{
"data": [
Expand Down Expand Up @@ -195,7 +195,7 @@ def _validate(
},
]
# </snippet>
# <snippet name="expect_queried_column_value_frequency_to_meet_threshold.py library_metadata">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py _validate function library_metadata">
# This dictionary contains metadata for display in the public gallery
library_metadata = {
"tags": ["query-based"],
Expand All @@ -205,7 +205,7 @@ def _validate(


if __name__ == "__main__":
# <snippet name="expect_queried_column_value_frequency_to_meet_threshold.py print_diagnostic_checklist()">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py print_diagnostic_checklist()">
ExpectQueriedColumnValueFrequencyToMeetThreshold().print_diagnostic_checklist()
# </snippet>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ def validate_configuration(
except AssertionError as e:
raise InvalidExpectationConfigurationError(str(e))

# <snippet name="expect_queried_table_row_count_to_be.py _validate function">
# <snippet name="expect_queried_table_row_count_to_be.py _validate function signature">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py _validate function">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py _validate function signature">
def _validate(
self,
metrics: dict,
Expand All @@ -81,7 +81,7 @@ def _validate(
}

# </snippet>
# <snippet name="expect_queried_table_row_count_to_be.py examples">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py examples">
examples = [
{
"data": [
Expand Down Expand Up @@ -144,16 +144,14 @@ def _validate(
]
# </snippet>
# This dictionary contains metadata for display in the public gallery
# <snippet name="expect_queried_table_row_count_to_be.py library_metadata">
library_metadata = {
"tags": ["query-based"],
"contributors": ["@joegargery"],
}
# </snippet>


if __name__ == "__main__":
# <snippet name="expect_queried_table_row_count_to_be.py print_diagnostic_checklist">
# <snippet name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py print_diagnostic_checklist">
ExpectQueriedTableRowCountToBe().print_diagnostic_checklist()
# </snippet>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ Make sure your one-line docstring begins with "Expect " and ends with a period.
```

with this one:
```python name="expect_queried_table_row_count_to_be.py print_diagnostic_checklist"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py print_diagnostic_checklist"
```

Later, you can go back and write a more thorough docstring. See [Expectation Docstring Formatting](https://github.com/great-expectations/great_expectations/blob/develop/docs/expectation_gallery/3-expectation-docstring-formatting.md).
Expand Down Expand Up @@ -159,7 +159,7 @@ You're going to search for `examples = []` in your file, and replace it with at

Your examples will look similar to this example:

```python name="expect_queried_table_row_count_to_be.py examples"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py examples"
```

Here's a quick overview of how to create test cases to populate `examples`. The overall structure is a list of dictionaries. Each dictionary has two keys:
Expand Down Expand Up @@ -259,7 +259,7 @@ In this step, we simply need to validate that the results of our Metrics meet ou

The validate method is implemented as `_validate(...)`:

```python name="expect_queried_table_row_count_to_be.py _validate function signature"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py _validate function signature"
```

This method takes a dictionary named `metrics`, which contains all Metrics requested by your Metric dependencies,
Expand All @@ -268,7 +268,7 @@ and performs a simple validation against your success keys (i.e. important thres
To do so, we'll be accessing our success keys, as well as the result of our previously-calculated Metrics.
For example, here is the definition of a `_validate(...)` method to validate the results of our `query.table` Metric against our success keys:

```python name="expect_queried_table_row_count_to_be.py _validate function"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_table_row_count_to_be.py _validate function"
```

Running your diagnostic checklist at this point should return something like this:
Expand Down Expand Up @@ -402,7 +402,7 @@ You'll also need to change the class name at the bottom of the file, by replacin
```

with this one:
```python name="expect_queried_column_value_frequency_to_meet_threshold.py print_diagnostic_checklist()"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py print_diagnostic_checklist()"
```

Later, you can go back and write a more thorough docstring.
Expand Down Expand Up @@ -436,7 +436,7 @@ Next, we're going to search for `examples = []` in your file, and replace it wit

Your examples will look something like this:

```python name="expect_queried_column_value_frequency_to_meet_threshold.py examples"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py examples"
```

Here's a quick overview of how to create test cases to populate `examples`. The overall structure is a list of dictionaries. Each dictionary has two keys:
Expand Down Expand Up @@ -539,7 +539,7 @@ In this step, we simply need to validate that the results of our Metrics meet ou

The validate method is implemented as `_validate(...)`:

```python name="expect_queried_column_value_frequency_to_meet_threshold.py _validate function signature"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py _validate function signature"
```

This method takes a dictionary named `metrics`, which contains all Metrics requested by your Metric dependencies,
Expand All @@ -548,7 +548,7 @@ and performs a simple validation against your success keys (i.e. important thres
To do so, we'll be accessing our success keys, as well as the result of our previously-calculated Metrics.
For example, here is the definition of a `_validate(...)` method to validate the results of our `query.column` Metric against our success keys:

```python name="expect_queried_column_value_frequency_to_meet_threshold.py _validate function"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py _validate function"
```

Running your diagnostic checklist at this point should return something like this:
Expand Down Expand Up @@ -616,7 +616,7 @@ If you plan to contribute your Expectation to the public open source project, yo

would become

```python name="expect_queried_column_value_frequency_to_meet_threshold.py library_metadata"
```python name="docs/docusaurus/docs/oss/guides/expectations/creating_custom_expectations/expect_queried_column_value_frequency_to_meet_threshold.py _validate function library_metadata"
```

This is particularly important because ***we*** want to make sure that ***you*** get credit for all your hard work!
Expand Down