Skip to content

Commit

Permalink
[DOCS] Update ci-cd order and fix link (#2514)
Browse files Browse the repository at this point in the history
  • Loading branch information
shir22 committed May 10, 2023
1 parent 411a25a commit 0dc9d7f
Show file tree
Hide file tree
Showing 3 changed files with 46 additions and 45 deletions.
83 changes: 42 additions & 41 deletions docs/source/general/usage/ci_cd.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,47 +64,6 @@ containing hand chosen checks and
:ref:`configure_check_conditions`
in order to cater to the specific needs of the project.

Airflow Integration
===================

.. image:: /_static/images/cicd/airflow.png
:alt: Airflow DAG example
:align: center

Airflow Quickstart
-------------------

Apache Airflow is an open-source workflow management system which is commonly used to automate data processing
pipelines.

If you are new to Airflow, you can get it up and running quickly with the following simplified steps:

1. Run ``pip install apache-airflow``
2. Run ``airflow standalone``. This will bootstrap a local Airflow deployment which is good for testing, but not intended for production
3. Under your home directory, insert your DAG (python file) inside the directory ``~/airflow/dags``

For a more elaborate explanation about installing and operating airflow see their
`docs <https://airflow.apache.org/docs/apache-airflow/stable/start.html>`__.

Airflow With Deepchecks
-----------------------

In the following example we will use S3 object storage to load the training data and to store our suite results. We define the first
2 tasks as short circuit tasks, which means the rest of the downstream tasks will be skipped if the return value of
them is false. This is useful in cases where we want to stop the pipeline if the data validation failed.
We can also add an additional step of deploying the model after the last validation has passed.

.. literalinclude:: ../../../../examples/cicd/airflow.py
:language: python
:tab-width: 0

We can access the result of the pipeline in our S3 bucket:

.. image:: /_static/images/cicd/s3_suites.png
:alt: Airflow DAG example
:align: center


GitHub Actions Integration
==========================

Expand Down Expand Up @@ -180,3 +139,45 @@ The yaml file for the GitHub Actions workflow is as follows:
with:
name: deepchecks results
path: *_validation.html
Airflow Integration
===================

.. image:: /_static/images/cicd/airflow.png
:alt: Airflow DAG example
:align: center

Airflow Quickstart
-------------------

Apache Airflow is an open-source workflow management system which is commonly used to automate data processing
pipelines.

If you are new to Airflow, you can get it up and running quickly with the following simplified steps:

1. Run ``pip install apache-airflow``
2. Run ``airflow standalone``. This will bootstrap a local Airflow deployment which is good for testing, but not intended for production
3. Under your home directory, insert your DAG (python file) inside the directory ``~/airflow/dags``

For a more elaborate explanation about installing and operating airflow see their
`docs <https://airflow.apache.org/docs/apache-airflow/stable/start.html>`__.

Airflow With Deepchecks
-----------------------

In the following example we will use S3 object storage to load the training data and to store our suite results. We define the first
2 tasks as short circuit tasks, which means the rest of the downstream tasks will be skipped if the return value of
them is false. This is useful in cases where we want to stop the pipeline if the data validation failed.
We can also add an additional step of deploying the model after the last validation has passed.

.. literalinclude:: ../../../../examples/cicd/airflow.py
:language: python
:tab-width: 0

We can access the result of the pipeline in our S3 bucket:

.. image:: /_static/images/cicd/s3_suites.png
:alt: Airflow DAG example
:align: center
2 changes: 1 addition & 1 deletion docs/source/getting-started/installation.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.. _installation_guide:
.. _getting-started__installation:

============
Installation
Expand Down
6 changes: 3 additions & 3 deletions docs/source/getting-started/welcome.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Get Started with Deepchecks Testing
:link-type: ref
:link: welcome__get_help

Links for how to interact with us via our `Slack Community <https://www.deepchecks.com/slack>`__
Links for how to interact with us via our `Slack Community <https://www.deepchecks.com/slack>`__
or by opening `an issue on Github <https://github.com/deepchecks/deepchecks/issues>`__.


Expand Down Expand Up @@ -179,11 +179,11 @@ Get Help & Give Us Feedback

In addition to perusing the documentation, feel free to:

- Ask questions on the `Slack Community <https://www.deepchecks.com/slack>`__,
- Ask questions on the `Slack Community <https://www.deepchecks.com/slack>`__.
- Post an issue or start a discussion on `Github Issues <https://github.com/deepchecks/deepchecks/issues>`__.
- To contribute to the package, check out the
`Contribution Guidelines <https://github.com/deepchecks/deepchecks/blob/main/CONTRIBUTING.rst>`__ and join the
`contributors-q-and-a Slack channel <https://deepcheckscommunity.slack.com/archives/C030REPARGR>`__,
`contributors-q-and-a channel <https://deepcheckscommunity.slack.com/archives/C030REPARGR>`__ on Slack,
or communicate with us via github issues.

To support us, please give us a star on ⭐️ `Github <https://github.com/deepchecks/deepchecks>`__ ⭐️,
Expand Down

0 comments on commit 0dc9d7f

Please sign in to comment.