Skip to content

Commit

Permalink
[docs] - Asset config [CON-89] (#8119)
Browse files Browse the repository at this point in the history
* First pass at asset config

* First pass at config example

* Move sections up

* Run scripts

* Remove recommendation

* Review comments and clean up

* Add import requests to example
  • Loading branch information
erinkcochran87 committed Jun 8, 2022
1 parent d759550 commit 2e236b2
Show file tree
Hide file tree
Showing 2 changed files with 42 additions and 11 deletions.
38 changes: 27 additions & 11 deletions docs/content/concepts/assets/software-defined-assets.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,33 @@ def downstream_asset(upstream_asset):

The [explicit dependencies](#explicit-dependencies) example covers an alternative way to specify asset dependencies without needing to match argument names to upstream asset names.

### Asset context

Since a software-defined asset contains an op, all the typical functionality of an op - like the use of [resources](/concepts/resources) and [configuration](#asset-configuration) - is available to an asset. Supplying the `context` parameter provides access to system information for the op, for example:

```python file=/concepts/assets/asset_w_context.py startafter=start_w_context endbefore=end_w_context
@asset(required_resource_keys={"api"})
def my_asset(context):
# fetches contents of an asset
return context.resources.api.fetch_table("my_asset")
```

### Asset configuration

Like ops, configuration is also supported for assets. Configuration is accessible through the asset context at runtime and can be used to specify behavior. Note that asset configuration behaves the same as configuration for ops.

For example, the following asset queries an API endpoint defined through configuration:

```python file=/concepts/assets/asset_config.py startafter=start_example endbefore=end_example
@asset(config_schema={"api_endpoint": str})
def my_configurable_asset(context):
api_endpoint = context.op_config["api_endpoint"]
data = requests.get(f"{api_endpoint}/data").json()
return data
```

Refer to the [Config schema documentation](/concepts/configuration/config-schema) for more configuration info and examples.

## Combining assets in groups

To materialize assets or load them in Dagit, you first need to combine them into an <PyObject object="AssetGroup" />, which is a set of assets with no unsatisfied dependencies. For example:
Expand Down Expand Up @@ -453,17 +480,6 @@ def cereal_asset():

Asset metadata can be viewed in Dagit on the [Asset Detail page](/concepts/dagit/dagit#asset-details).

### Using context in assets

Since a software-defined asset contains an op, all of the typical functionality of an op (like the use of resources) is available to an asset.

```python file=/concepts/assets/asset_w_context.py startafter=start_w_context endbefore=end_w_context
@asset(required_resource_keys={"api"})
def my_asset(context):
# fetches contents of an asset
return context.resources.api.fetch_table("my_asset")
```

## Further Reading

Interested in learning more about software-defined assets and working through a more complex example? Check out our [guide on software-defined assets](/guides/dagster/software-defined-assets) and our [example project](https://github.com/dagster-io/dagster/tree/master/examples/modern_data_stack_assets) that integrates software-defined assets with other Modern Data Stack tools.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import requests

from dagster import asset

# start_example


@asset(config_schema={"api_endpoint": str})
def my_configurable_asset(context):
api_endpoint = context.op_config["api_endpoint"]
data = requests.get(f"{api_endpoint}/data").json()
return data


# end_example

1 comment on commit 2e236b2

@vercel
Copy link

@vercel vercel bot commented on 2e236b2 Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.