/
ticketswap.yml
169 lines (169 loc) · 6.26 KB
/
ticketswap.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
capabilities:
- about
- stream-maps
- schema-flattening
- target-schema
- hard-delete
description: Amazon Redshift loader
domain_url: https://aws.amazon.com/redshift/
executable: target-redshift
keywords:
- meltano_sdk
- database
label: Amazon Redshift
logo_url: /assets/logos/loaders/redshift.png
maintenance_status: active
name: target-redshift
namespace: target_redshift
next_steps: ''
pip_url: git+https://github.com/TicketSwap/target-redshift.git
quality: silver
repo: https://github.com/TicketSwap/target-redshift
settings:
- description: If set to false, the tap will ignore activate version messages. If
set to true, add_record_metadata must be set to true as well.
kind: boolean
label: Activate Version
name: activate_version
value: true
- description: Note that this must be enabled for activate_version to work!This adds
_sdc_extracted_at, _sdc_batched_at, and more to every table. See https://sdk.meltano.com/en/latest/implementation/record_metadata.html
for more information.
kind: boolean
label: Add Record Metadata
name: add_record_metadata
value: true
- description: Database name. Note if sqlalchemy_url is set this will be ignored.
kind: string
label: Database name
name: dbname
- description: 'Redshift schema to send data to, example: tap-clickup'
kind: string
label: Default Target Schema
name: default_target_schema
value: $MELTANO_EXTRACT__LOAD_SCHEMA
- description: Dialect+driver see https://aws.amazon.com/blogs/big-data/use-the-amazon-redshift-sqlalchemy-dialect-to-interact-with-amazon-redshift.
Generally just leave this alone. Note if sqlalchemy_url is set this will be ignored.
kind: string
label: Dialect+Driver
name: dialect+driver
value: redshift+redshift_connector
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
label: Flattening Enabled
name: flattening_enabled
- description: The max depth to flatten schemas.
kind: integer
label: Flattening Max Depth
name: flattening_max_depth
- description: When activate version is sent from a tap this specefies if we should
delete the records that don't match, or mark them with a date in the `_sdc_deleted_at`
column. This config option is ignored if `activate_version` is set to false.
kind: boolean
label: Hard Delete
name: hard_delete
value: false
- description: Hostname for redshift instance. Note if sqlalchemy_url is set this
will be ignored.
kind: string
label: Host
name: host
- description: If set to true, the target will interpret the content encoding of the
schema to determine how to store the data. Using this option may result in a more
efficient storage of the data but may also result in an error if the data is not
encoded as expected.
kind: boolean
label: Interpret Content Encoding
name: interpret_content_encoding
value: false
- description: Password used to authenticate. Note if sqlalchemy_url is set this will
be ignored.
kind: password
label: Password
name: password
sensitive: true
- description: The port on which redshift is awaiting connection. Note if sqlalchemy_url
is set this will be ignored.
kind: string
label: Port
name: port
value: "5432"
- description: SQLAlchemy connection string. This will override using host, user,
password, port, dialect, and all ssl settings. Note that you must escape password
special characters properly. See
https://docs.sqlalchemy.org/en/20/core/engines.html#escaping-special-characters-such-as-signs-in-passwords
kind: string
label: Sqlalchemy URL
name: sqlalchemy_url
- description: Username to connect to bastion host
kind: string
label: SSH Tunnel Username
name: ssh_tunnel.username
- description: Whether or not to use ssl to verify the server's identity. Use ssl_certificate_authority
and ssl_mode for further customization. To use a client certificate to authenticate
yourself to the server, use ssl_client_certificate_enable instead. Note if sqlalchemy_url
is set this will be ignored.
kind: boolean
label: SSL Enable
name: ssl_enable
value: false
- description: SSL Protection method. Must be one of disable, allow, prefer, require, verify-ca,
or verify-full. Note if sqlalchemy_url is set this will be ignored.
kind: string
label: SSL Mode
name: ssl_mode
value: verify-full
- description: User-defined config values to be used within map expressions.
kind: object
label: Stream Map Config
name: stream_map_config
- description: Config object for stream maps capability. For more information check
out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html).
kind: object
label: Stream Maps
name: stream_maps
- description: User name used to authenticate. Note if sqlalchemy_url is set this
will be ignored.
kind: string
label: User
name: user
- description: If true, use temporary credentials
(https://docs.aws.amazon.com/redshift/latest/mgmt/generating-iam-credentials-cli-api.html).
Note if sqlalchemy_url is set this will be ignored.
kind: boolean
label: Enable iam authentication
name: enable_iam_authentication
value: false
- description: Redshift cluster identifier. Note if sqlalchemy_url is set or enable_iam_authentication is false this will be ignored.
kind: string
label: Cluster identifier
name: cluster_identifier
- description: Redshift copy role arn to use for the COPY command from s3.
kind: string
label: AWS redshift copy role arn
name: aws_redshift_copy_role_arn
- description: S3 bucket to save staging files before using COPY command.
kind: string
label: S3 bucket
name: s3_bucket
- description: S3 key prefix to save staging files before using COPY command.
kind: string
label: S3 key prefix
name: s3_key_prefix
- description: If you want to remove staging files in S3.
kind: boolean
label: Remove S3 files
name: remove_s3_files
value: true
- description: Where you want to store your staging files locally.
kind: string
label: Temp dir
name: temp_dir
value: .meltano/temp
settings_group_validation:
- []
settings_preamble: 'This target uses boto3 to connect to AWS and perform copy and possibly credential operations. The agent running this target should be connected to AWS to be able to open a boto3 client.'
target_schema: $TARGET_REDSHIFT_SCHEMA
usage: ''
variant: ticketswap