Skip to content

Latest commit

 

History

History
47 lines (38 loc) · 1.92 KB

append.rst

File metadata and controls

47 lines (38 loc) · 1.92 KB
Raw

append operator

When to use the append operator

The append operator allows you to append data from a source table to a target table.

../../../../example_dags/example_append.py

If a columns parameter is not provided, the append operator assumes that the source and target tables have the same schema.

Tables have the same schema

  1. Case 1: When the complete table needs to be appended, the columns parameter can be omitted.

    ../../../../example_dags/example_append.py

  2. Case 2: When only a subset of columns needs to be appended, you can pass a list of columns to the columns parameter.

    ../../../../example_dags/example_snowflake_partial_table_with_append.py

Tables have different schemas

When the source and target tables have different schemas, you can map different column names by passing a dict of source columns to target columns.

../../../../example_dags/example_append.py

Conflicts

The append operator can't handle conflicts that may arise while appending data. If you want to handle those scenarios, you can use the merge_operator.

Default Datasets

  • Input dataset - Source table for the operator.
  • Output dataset - Target table of the operator.