Skip to content

Commit

Permalink
replace EventMetadata in docs (#6774)
Browse files Browse the repository at this point in the history
* replace EventMetadata in docs
  • Loading branch information
smackesey committed Feb 24, 2022
1 parent f470ea9 commit b26ba51
Show file tree
Hide file tree
Showing 5 changed files with 10 additions and 14 deletions.
6 changes: 3 additions & 3 deletions docs/content/concepts/assets/asset-materializations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ class PandasCsvIOManager(IOManager):

### Attaching Metadata to an AssetMaterialization

There are a variety of types of metadata that can be associated with a materialization event, all through the <PyObject module="dagster" object="EventMetadataEntry" /> class. Each materialization event optionally takes a list of metadata entries that are then displayed in the event log and the [Asset Catalog](/concepts/dagit/dagit#assets).
There are a variety of types of metadata that can be associated with a materialization event, all through the <PyObject module="dagster" object="MetadataEntry" /> class. Each materialization event optionally takes a list of metadata entries that are then displayed in the event log and the [Asset Catalog](/concepts/dagit/dagit#assets).

#### Example: Op body

Expand Down Expand Up @@ -147,7 +147,7 @@ class PandasCsvIOManagerWithAsset(IOManager):
)
```

Check our API docs for <PyObject module="dagster" object="EventMetadataEntry" /> for more details on they types of event metadata available.
Check our API docs for <PyObject module="dagster" object="MetadataEntry" /> for more details on they types of event metadata available.

### Specifying a partition for an AssetMaterialization

Expand Down Expand Up @@ -228,7 +228,7 @@ class PandasCsvIOManagerWithOutputAsset(IOManager):
return AssetKey(file_path)
```

When an output is linked to an asset in this way, the generated <PyObject object="AssetMaterialization" /> event will contain any <PyObject object="EventMetadataEntry" /> information yielded from the `handle_output` function (in addiition to all of the `metadata` specified on the corresponding <PyObject object="Output" /> event).
When an output is linked to an asset in this way, the generated <PyObject object="AssetMaterialization" /> event will contain any <PyObject object="MetadataEntry" /> information yielded from the `handle_output` function (in addiition to all of the `metadata` specified on the corresponding <PyObject object="Output" /> event).

See the [IOManager docs](/concepts/io-management/io-managers#yielding-metadata-from-an-iomanager) for more information on yielding these entries from an IOManager.

Expand Down
2 changes: 1 addition & 1 deletion docs/content/concepts/assets/asset-observations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ height={917}

### Attaching Metadata to an AssetObservation

There are a variety of types of metadata that can be associated with an observation event, all through the <PyObject object="EventMetadataEntry" /> class. Each observation event optionally takes a dictionary of metadata entries that are then displayed in the event log and the [Asset Details](/concepts/dagit/dagit#asset-details) page. Check our API docs for <PyObject object="EventMetadataEntry" /> for more details on the types of event metadata available.
There are a variety of types of metadata that can be associated with an observation event, all through the <PyObject object="MetadataEntry" /> class. Each observation event optionally takes a dictionary of metadata entries that are then displayed in the event log and the [Asset Details](/concepts/dagit/dagit#asset-details) page. Check our API docs for <PyObject object="MetadataEntry" /> for more details on the types of event metadata available.

```python file=concepts/assets/observations.py startafter=start_observation_asset_marker_2 endbefore=end_observation_asset_marker_2
from dagster import op, AssetObservation, Output, EventMetadata
Expand Down
2 changes: 1 addition & 1 deletion docs/content/concepts/io-management/io-managers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ def test_my_io_manager_load_input():

## Yielding metadata from an IOManager <Experimental />

Sometimes, you may want to record some metadata while handling an output in an IOManager. To do this, you can optionally yield <PyObject object="EventMetadataEntry"/> objects from within the body of the `handle_output` function. Using this, we can modify one of the [above examples](/concepts/io-management/io-managers#a-custom-io-manager-that-stores-pandas-dataframes-in-tables) to now include some helpful metadata in the log:
Sometimes, you may want to record some metadata while handling an output in an IOManager. To do this, you can optionally yield <PyObject object="MetadataEntry"/> objects from within the body of the `handle_output` function. Using this, we can modify one of the [above examples](/concepts/io-management/io-managers#a-custom-io-manager-that-stores-pandas-dataframes-in-tables) to now include some helpful metadata in the log:

```python file=/concepts/io_management/custom_io_manager.py startafter=start_metadata_marker endbefore=end_metadata_marker
class DataframeTableIOManagerWithMetadata(IOManager):
Expand Down
12 changes: 4 additions & 8 deletions docs/content/concepts/ops-jobs-graphs/op-events.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,9 @@ It is also possible to raise Dagster-specific exceptions, which will indicate to

### Event Metadata

Often, it may be useful to attach some arbitrary information to an event or exception that is not captured by its basic parameters. Through the <PyObject object="EventMetadataEntry"/> object, we provide a consistent interface for specifying this metadata on a variety of events. Depending on the type of the data, these entries will be rendered in Dagit in a more useful format than a simple unstructured string.
Often, it may be useful to attach some arbitrary information to an event or exception that is not captured by its basic parameters. Through the <PyObject object="MetadataEntry"/> and <PyObject object="MetadataValue"/> classes, we provide a consistent interface for specifying this metadata as a set of "entries". Individual entries are instances of the `MetadataEntry` class, which is a simple container for a string `label` and a typed `value`. The available value types are accessible through a static API defined on `MetadataValue`. These include simple datatypes (`MetadataValue.float`, `MetadataValue.int`, `MetadataValue.text`), as well as more complex information such as markdown and json (`MetadataValue.md`, `MetadataValue.json`). Depending on the type of its `value`, a `MetadataEntry` will be rendered in Dagit in a more useful format than a simple unstructured string.

The <PyObject object="AssetMaterialization" />, <PyObject object="AssetObservation" />, <PyObject object="ExpectationResult" />, and <PyObject object="Failure" /> objects each accept a `metadata` parameter, which maps string labels to structured values. <PyObject object="Output"/> also accepts this parameter, although this functionality is currently experimental may change in the future.

We provide support for a wide variety of potentially useful metadata types, including simple datatypes (`EventMetadataEntry.float`, `EventMetadataEntry.int`, `EventMetadataEntry.text`), as well as more complex information such as markdown and json (`EventMetadataEntry.md`, `EventMetadataEntry.json`).

Check out the API Docs for <PyObject object="EventMetadataEntry"/> for more details.
Metadata is attached to events at construction time. There are currently two available metadata specification keyword arguments on all event constructors: (1) `metadata` takes a dictionary mapping string `labels` to `MetadataValue` objects. Under the hood, this is resolved to a list of `MetadataEntry` objects. (2) `metadata_entries` takes a list of `MetadataEntry` objects directly. The `metadata_entries` argument is deprecated-- new code should use the `metadata` dictionary form.

## Events

Expand Down Expand Up @@ -178,7 +174,7 @@ def my_expectation_op(context, df):

#### Attaching Metadata to Expectation Results

Like many other event types in Dagster, there are a variety of types of metadata that can be associated with an expectation result event, all through the <PyObject object="EventMetadataEntry"/> class. Each expectation event optionally takes a list of metadata entries that are then displayed in the event log.
Like many other event types in Dagster, there are a variety of types of metadata that can be associated with an expectation result event, all through the <PyObject object="MetadataEntry"/> class. Each expectation event optionally takes a list of metadata entries that are then displayed in the event log.

This example shows metadata entries of different types attached to the same expectation result:

Expand Down Expand Up @@ -228,7 +224,7 @@ def my_failure_op():

#### Attaching Metadata to Failures

As is the case with many Dagster-provided classes, you can attach a list <PyObject object="EventMetadataEntry"/> denoting an arbitrary set of metadata relevant to the failure.
As is the case with many Dagster-provided classes, you can attach a list <PyObject object="MetadataEntry"/> denoting an arbitrary set of metadata relevant to the failure.

```python file=/concepts/solids_pipelines/op_events.py startafter=start_failure_metadata_op endbefore=end_failure_metadata_op
@op
Expand Down
2 changes: 1 addition & 1 deletion docs/content/integrations/pandas.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ If we rerun the above example with this dataframe, nothing should change. Howeve

## Dagster DataFrame Summary Statistics

Aside from constraint validation, `create_dagster_pandas_dataframe_type` also takes in a summary statistics function that emits `EventMetadataEntry` objects which are surfaced during runs. Since data systems seldom control the quality of the data they receive, it becomes important to monitor data as it flows through your systems. In complex jobs, this can help debug and monitor data drift over time. Let's illustrate how this works in our example:
Aside from constraint validation, `create_dagster_pandas_dataframe_type` also takes in a summary statistics function that emits `MetadataEntry` objects which are surfaced during runs. Since data systems seldom control the quality of the data they receive, it becomes important to monitor data as it flows through your systems. In complex jobs, this can help debug and monitor data drift over time. Let's illustrate how this works in our example:

```python file=/legacy/dagster_pandas_guide/summary_stats.py startafter=start_summary endbefore=end_summary
def compute_trip_dataframe_summary_statistics(dataframe):
Expand Down

1 comment on commit b26ba51

@vercel
Copy link

@vercel vercel bot commented on b26ba51 Feb 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.