Skip to content

Commit

Permalink
[docs] - Asset grouping [CON-295] (#8375)
Browse files Browse the repository at this point in the history
* First pass at asset group docs

* Updates

* Running snapshot

* Update placeholder image

* Replace placeholder image

* Review comments

* Remove unused imports
  • Loading branch information
erinkcochran87 authored and clairelin135 committed Jun 14, 2022
1 parent 662f47a commit 0889686
Show file tree
Hide file tree
Showing 5 changed files with 97 additions and 0 deletions.
67 changes: 67 additions & 0 deletions docs/content/concepts/assets/software-defined-assets.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,73 @@ def repo():

Like regular jobs, asset jobs can be placed on [schedules](/concepts/partitions-schedules-sensors/schedules) and [sensors](/concepts/partitions-schedules-sensors/sensors).

---

## Grouping assets

To help keep your assets tidy, you can organize them into groups. Grouping assets by project, concept, and so on simplifies [keeping track of them in Dagit](#viewing-asset-groups-in-dagit).

- [Assigning assets to groups](#assigning-assets-to-groups)
- [Viewing asset groups in Dagit](#viewing-asset-groups-in-dagit)

### Assigning assets to groups

In Dagster, there are two ways to assign assets to groups:

- [By using the group_name argument when calling `load_assets_from_package_module`](#from-assets-in-a-sub-module) (**recommended**)
- [By specifying a group name on an individual asset](#on-individual-assets)

By default, assets that aren't assigned to a group will be placed in a group named `default`. Use Dagit to [view these assets](#viewing-asset-groups-in-dagit).

#### From assets in a sub-module

**This recommended approach** constructs a group of assets from a specified module in your project. Using the `load_assets_from_package_module` function, you can import all assets in a module and apply a grouping:

```python file=/concepts/assets/asset_group_module.py startafter=start_example endbefore=end_example
import my_package.cereal as cereal

cereal_assets = load_assets_from_package_module(
cereal,
group_name="cereal_assets",
)
```

#### On individual assets

Assets can also be given groups on an individual basis by specifying an argument when creating the asset:

```python file=/concepts/assets/asset_group_argument.py startafter=start_example endbefore=end_example
@asset(group_name="cereal_assets")
def nabisco_cereals():
return [1, 2, 3]
```

#### To multiple groups

**Assets can only be assigned to one group at a time**. Attempting to place a grouped asset in a second group will result in an error:

```bash
Group name already exists on assets [list_of_asset_keys]
```

### Viewing asset groups in Dagit

To view your asset groups in Dagit, open the left navigation by clicking the **menu icon in the top left corner**. As asset groups are grouped in repositories, you may need to open a repository to view its asset groups:

<img
alt="Asset Groups in Dagit left navigation"
src="/images/concepts/assets/asset-groups-in-sidenav.png"
/>

Click the asset group to open a dependency graph for all assets in the group:

<img
alt="Dependency graph for an asset group"
src="/images/concepts/assets/asset-group-dependency-graph.png"
/>

---

## Testing

When writing unit tests, you can treat the function decorated by `@asset` as a regular python function.
Expand Down
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# pylint: disable=redefined-outer-name

from dagster import asset

# start_example


@asset(group_name="cereal_assets")
def nabisco_cereals():
return [1, 2, 3]


# end_example
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# isort: skip_file
# pylint: disable=

from dagster import (
load_assets_from_package_module,
)

# start_example

import my_package.cereal as cereal

cereal_assets = load_assets_from_package_module(
cereal,
group_name="cereal_assets",
)

# end_example

0 comments on commit 0889686

Please sign in to comment.