Add documentation for disabling snapshots #115
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vsreekanti
approved these changes
Jun 7, 2023
artifacts.md
Outdated
|
|
||
|
|
||
| ## Disabling Snapshots on Artifacts | ||
| By default, Aqueduct snapshots all artifact results during workflow runs. You can change this setting when [publishing the workflow](./workflows/creating-a-workflow.md), or call `.disable_snapshot()` or `.enable_snapshot()` on individual artifacts. For example: |
There was a problem hiding this comment.
Tiny nit: " when publishing the workflow by setting the disable_snapshots flag to True. You can also call .disable_snapshot():"
Then after the code snippet, add:
On an artifact that has had snapshotting disabled, you can call
enable_snapshot()to reenable snapshots.
workflows/creating-a-workflow.md
Outdated
| @@ -88,6 +88,7 @@ There are a few key arguments here, and we'll go through the one by one: | |||
| * You're, of course, welcome to list out all of the data artifacts in your workflow, but we figured it would be easier to list the outputs you care about. | |||
| * `schedule`: This tells us how often you'd like to run your workflow. If you leave this empty, no schedule will be set, and you can set a schedule that executes as quickly as every minute or as rarely as every month. (See [managing-workflow-schedules.md](managing-workflow-schedules.md "mention") for more details.) | |||
| * `config`: This tells us which connected engine you'd like to run your workflow. If you leave this empty, the workflow will be executed by Aqueduct engine by default. We currently support Airflow, Kubernetes, and Lambda. | |||
| * `disable_snapshots`: This tells us if we should store snapshots of intermediate results. This disables all snapshots NOT coming from parameters, metrics, or checks. Additionally, you can alternate this behavior on individual [artifacts](../artifacts.md) by calling `.disable_snapshot()` on the artifact. | |||
There was a problem hiding this comment.
This disables all snapshots except for parameter, metrics, or checks.
Instead of "alternate", let's use "toggle".
workflows/workflow-versions.md
Outdated
| @@ -6,7 +6,7 @@ Workflows are versioned on every execution. You can see previous versions on by | |||
|
|
|||
| On every execution, Aqueduct automatically captures the code that was executed as well as _all the data that was generated by the workflow_ -- this includes intermediary data artifacts that aren't explicitly saved by the workflow. Aqueduct captures this information on each workflow run in order to help debug workflows, both within an individual execution and across time.  | |||
|
|
|||
| By default, the snapshots of intermediary data are stored on your local filesystem. **NOTE**: We are working on making this feature configurable. | |||
| By default, the snapshots of intermediary data are stored on your local filesystem. You can alternate this settings when publishing the workflow. | |||
vsreekanti
approved these changes
Jun 7, 2023
|
GH failure due to server mismatch during release, confirmed low risk |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.