Skip to content

Commit

Permalink
asset apiref improvements (#8374)
Browse files Browse the repository at this point in the history
  • Loading branch information
sryza committed Jun 14, 2022
1 parent 42d4903 commit a20b02b
Show file tree
Hide file tree
Showing 7 changed files with 14 additions and 24 deletions.
4 changes: 2 additions & 2 deletions docs/content/_apidocs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Dagster follows [SemVer](https://semver.org/). We attempt to isolate breaking ch

APIs from the core `dagster` package, divided roughly by topic:

- [Software-Defined Assets](/\_apidocs/assets). An asset is an object in persistent storage, such as a table, file, or persisted machine learning model. A software-defined asset is a Dagster object that couples an asset to the function and upstream assets that are used to produce its contents.

- [Jobs](/\_apidocs/jobs) APIs to define jobs that execute a set of ops with specific parameters.

- [Ops](/\_apidocs/ops) APIs to define or decorate functions as ops, declare their inputs and outputs, compose ops with each other, as well as the datatypes that op execution can return or yield.
Expand Down Expand Up @@ -46,8 +48,6 @@ APIs from the core `dagster` package, divided roughly by topic:

- [Versioning & Memoization](/\_apidocs/memoization) (Experimental) Code versioning and memoization of previous outputs based upon that versioning.

- [Software-Defined Assets](/\_apidocs/assets) (Experimental). Software-defined assets sit on top of the graph/job/op APIs and enable a novel way of constructing Dagster jobs that puts assets at the forefront.

- [Utilities](/\_apidocs/utilities) Miscellaneous helpers used by Dagster that may be useful to users.

- [Internals](/\_apidocs/internals) Core internal APIs that are important if you are interested in understanding how Dagster works with an eye towards extending it: logging, executors, system storage, the Dagster instance & plugin machinery, storage, schedulers.
Expand Down
8 changes: 4 additions & 4 deletions docs/content/_navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,10 @@
{
"title": "Core",
"children": [
{
"title": "Software-Defined Assets",
"path": "/_apidocs/assets"
},
{
"title": "Jobs",
"path": "/_apidocs/jobs"
Expand Down Expand Up @@ -494,10 +498,6 @@
"title": "Versioning & Memoization",
"path": "/_apidocs/memoization"
},
{
"title": "Software-Defined Assets",
"path": "/_apidocs/assets"
},
{
"title": "Utilities",
"path": "/_apidocs/utilities"
Expand Down
2 changes: 1 addition & 1 deletion docs/content/api/modules.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/content/api/searchindex.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/content/api/sections.json

Large diffs are not rendered by default.

Binary file modified docs/next/public/objects.inv
Binary file not shown.
20 changes: 5 additions & 15 deletions docs/sphinx/sections/api/apidocs/assets.rst
Original file line number Diff line number Diff line change
@@ -1,30 +1,20 @@
.. currentmodule:: dagster

Software-Defined Assets (Experimental)
======================================
Software-Defined Assets
=======================

Software-defined assets sit on top of the graph/job/op APIs and enable a novel way of constructing Dagster jobs that puts assets at the forefront.

Conceptually, software-defined assets invert the typical relationship between assets and computation. Instead of defining a graph of ops and recording which assets those ops end up materializing, you define a set of assets, each of which knows how to compute its contents from upstream assets.

A software-defined asset combines:
- An asset key, e.g. the name of a table.
- A function, which can be run to compute the contents of the asset.
- A set of upstream assets that are provided as inputs to the function when computing the asset.
An asset is an object in persistent storage, such as a table, file, or persisted machine learning model. A software-defined asset is a Dagster object that couples an asset to the function and upstream assets that are used to produce its contents.

.. autodecorator:: asset

.. autoclass:: AssetGroup
:members:

.. autodecorator:: multi_asset

.. autofunction:: build_assets_job

.. autoclass:: AssetIn

.. autoclass:: SourceAsset

.. autofunction:: define_asset_job

.. autofunction:: load_assets_from_modules

.. autofunction:: load_assets_from_current_module
Expand Down

1 comment on commit a20b02b

@vercel
Copy link

@vercel vercel bot commented on a20b02b Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.