-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Possibly missing some configuration option #9
Comments
Here is my log when running 2023-09-26T08:27:50.970629Z [info ] Running extract & load... name=meltano run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl
2023-09-26T08:27:51.584625Z [warning ] No state was found, complete import.
2023-09-26T08:28:18.406789Z [info ] 2023-09-26 03:28:18,406 | INFO | tap-shopify | Skipping deselected stream 'automatic_discount_nodes'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.407255Z [info ] 2023-09-26 03:28:18,406 | INFO | tap-shopify | Skipping deselected stream 'catalogs'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.407624Z [info ] 2023-09-26 03:28:18,406 | INFO | tap-shopify | Skipping deselected stream 'checkout_profiles'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.407851Z [info ] 2023-09-26 03:28:18,406 | INFO | tap-shopify | Skipping deselected stream 'code_discount_nodes'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408037Z [info ] 2023-09-26 03:28:18,406 | INFO | tap-shopify | Skipping deselected stream 'collections'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408140Z [info ] 2023-09-26 03:28:18,407 | INFO | tap-shopify | Skipping deselected stream 'companies'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408355Z [info ] 2023-09-26 03:28:18,407 | INFO | tap-shopify | Skipping deselected stream 'company_locations'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408462Z [info ] 2023-09-26 03:28:18,408 | INFO | tap-shopify | Skipping deselected stream 'customers'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408558Z [info ] 2023-09-26 03:28:18,408 | INFO | tap-shopify | Skipping deselected stream 'deletion_events'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408732Z [info ] 2023-09-26 03:28:18,408 | INFO | tap-shopify | Skipping deselected stream 'delivery_customizations'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408830Z [info ] 2023-09-26 03:28:18,408 | INFO | tap-shopify | Skipping deselected stream 'discount_nodes'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.408922Z [info ] 2023-09-26 03:28:18,408 | INFO | tap-shopify | Skipping deselected stream 'discount_redeem_code_saved_searches'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409073Z [info ] 2023-09-26 03:28:18,408 | INFO | tap-shopify | Skipping deselected stream 'draft_orders'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409171Z [info ] 2023-09-26 03:28:18,408 | INFO | tap-shopify | Skipping deselected stream 'files'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409408Z [info ] 2023-09-26 03:28:18,409 | INFO | tap-shopify | Skipping deselected stream 'fulfillment_orders'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409562Z [info ] 2023-09-26 03:28:18,409 | INFO | tap-shopify | Skipping deselected stream 'gift_cards'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409659Z [info ] 2023-09-26 03:28:18,409 | INFO | tap-shopify | Skipping deselected stream 'inventory_items'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409749Z [info ] 2023-09-26 03:28:18,409 | INFO | tap-shopify | Skipping deselected stream 'locations'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409855Z [info ] 2023-09-26 03:28:18,409 | INFO | tap-shopify | Skipping deselected stream 'manual_holds_fulfillment_orders'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.409994Z [info ] 2023-09-26 03:28:18,409 | INFO | tap-shopify | Skipping deselected stream 'marketing_activities'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.410125Z [info ] 2023-09-26 03:28:18,409 | INFO | tap-shopify | Skipping deselected stream 'marketing_events'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.410265Z [info ] 2023-09-26 03:28:18,410 | INFO | tap-shopify | Skipping deselected stream 'metafield_definitions'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.410433Z [info ] 2023-09-26 03:28:18,410 | INFO | tap-shopify | Skipping deselected stream 'metaobjects'. cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.412148Z [info ] 2023-09-26 03:28:18,412 | INFO | tap-shopify | Beginning incremental sync of 'orders'... cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:18.412303Z [info ] 2023-09-26 03:28:18,412 | INFO | tap-shopify | Tap has custom mapper. Using 1 provided map(s). cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.155387Z [info ] 2023-09-26 03:28:19,155 | INFO | singer_sdk.metrics | METRIC: {"type": "counter", "metric": "http_request_count", "value": 0, "tags": {"stream": "orders", "endpoint": ""}} cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.155659Z [info ] 2023-09-26 03:28:19,155 | INFO | singer_sdk.metrics | METRIC: {"type": "timer", "metric": "sync_duration", "value": 0.40765810012817383, "tags": {"stream": "orders", "context": {}, "status": "failed"}} cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.155922Z [info ] 2023-09-26 03:28:19,155 | INFO | singer_sdk.metrics | METRIC: {"type": "counter", "metric": "record_count", "value": 0, "tags": {"stream": "orders", "context": {}}} cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.156254Z [info ] 2023-09-26 03:28:19,155 | ERROR | tap-shopify | An unhandled error occurred while syncing 'orders' cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.156431Z [info ] Traceback (most recent call last): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.156585Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1175, in sync cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.156743Z [info ] for _ in self._sync_records(context=context): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.156877Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1070, in _sync_records cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157016Z [info ] for record_result in self.get_records(current_context): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157147Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 576, in get_records cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157279Z [info ] for record in self.request_records(context): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157407Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 393, in request_records cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157543Z [info ] prepared_request = self.prepare_request( cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157672Z [info ] ^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157800Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 364, in prepare_request cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.157951Z [info ] request_data = self.prepare_request_payload(context, next_page_token) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158086Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158213Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 365, in prepare_request_payload cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158342Z [info ] query = self.query.lstrip() cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158470Z [info ] ^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158597Z [info ] File "/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py", line 1001, in __get__ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158724Z [info ] val = self.func(instance) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158851Z [info ] ^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.158977Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 497, in query cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.159106Z [info ] self.evaluate_query() cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.159235Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 521, in evaluate_query cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.159366Z [info ] error_code = error.get("extensions", {}).get("code") cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.159500Z [info ] ^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.159632Z [info ] AttributeError: 'str' object has no attribute 'get' cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.159839Z [info ] Traceback (most recent call last): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.160145Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/bin/tap-shopify", line 8, in <module> cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.160287Z [info ] sys.exit(TapShopify.cli()) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.160426Z [info ] ^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.160555Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.160688Z [info ] return self.main(*args, **kwargs) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.160819Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.160945Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161079Z [info ] rv = self.invoke(ctx) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161207Z [info ] ^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161334Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161465Z [info ] return ctx.invoke(self.callback, **ctx.params) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161592Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161718Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161849Z [info ] return __callback(*args, **kwargs) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.161978Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.162124Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/tap_base.py", line 498, in invoke cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.162258Z [info ] tap.sync_all() cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.162385Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/tap_base.py", line 456, in sync_all cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.162515Z [info ] stream.sync() cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.162641Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1182, in sync cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.162770Z [info ] raise ex cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.162898Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1175, in sync cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.163109Z [info ] for _ in self._sync_records(context=context): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.163277Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1070, in _sync_records cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.163472Z [info ] for record_result in self.get_records(current_context): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.163612Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 576, in get_records cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.163750Z [info ] for record in self.request_records(context): cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.163884Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 393, in request_records cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.164043Z [info ] prepared_request = self.prepare_request( cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.164177Z [info ] ^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.164308Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 364, in prepare_request cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.165142Z [info ] request_data = self.prepare_request_payload(context, next_page_token) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.165310Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.165454Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 365, in prepare_request_payload cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.165594Z [info ] query = self.query.lstrip() cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.165730Z [info ] ^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.165864Z [info ] File "/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py", line 1001, in __get__ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.165996Z [info ] val = self.func(instance) cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.166125Z [info ] ^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.166260Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 497, in query cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.166508Z [info ] self.evaluate_query() cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.166652Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 521, in evaluate_query cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.166788Z [info ] error_code = error.get("extensions", {}).get("code") cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.166946Z [info ] ^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:19.167079Z [info ] AttributeError: 'str' object has no attribute 'get' cmd_type=extractor name=tap-shopify run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:20.086857Z [info ] Incremental state has been updated at 2023-09-26 08:28:20.086689.
2023-09-26T08:28:20.095897Z [error ] Extraction failed code=1 message=AttributeError: 'str' object has no attribute 'get' name=meltano run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl
2023-09-26T08:28:20.096407Z [info ] ELT could not be completed: Extractor failed. cmd_type=elt name=meltano run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:20.096562Z [info ] For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag. cmd_type=elt name=meltano run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:20.096698Z [info ] Note that you can also check the generated log file at '/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/logs/elt/2023-09-26T082750--tap-shopify--target-jsonl/e9709048-99fb-4aa8-8c6c-6ae1bc2d011a/elt.log'. cmd_type=elt name=meltano run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr
2023-09-26T08:28:20.096829Z [info ] For more information on debugging and logging: https://docs.meltano.com/reference/command-line-interface#debugging cmd_type=elt name=meltano run_id=e9709048-99fb-4aa8-8c6c-6ae1bc2d011a state_id=2023-09-26T082750--tap-shopify--target-jsonl stdio=stderr |
And here is me trying to run 2023-09-26T08:37:13.329392Z [info ] Running extract & load... name=meltano run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres
2023-09-26T08:37:14.090548Z [warning ] No state was found, complete import.
2023-09-26T08:37:32.122614Z [info ] 2023-09-26 03:37:32,122 | INFO | target-postgres | Target 'target-postgres' is listening for input from tap. cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.920422Z [info ] 2023-09-26 03:37:42,920 | INFO | tap-shopify | Skipping deselected stream 'automatic_discount_nodes'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.920817Z [info ] 2023-09-26 03:37:42,920 | INFO | tap-shopify | Skipping deselected stream 'catalogs'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.921016Z [info ] 2023-09-26 03:37:42,920 | INFO | tap-shopify | Skipping deselected stream 'checkout_profiles'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.921172Z [info ] 2023-09-26 03:37:42,920 | INFO | tap-shopify | Skipping deselected stream 'code_discount_nodes'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.921456Z [info ] 2023-09-26 03:37:42,920 | INFO | tap-shopify | Skipping deselected stream 'collections'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.921672Z [info ] 2023-09-26 03:37:42,920 | INFO | tap-shopify | Skipping deselected stream 'companies'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.922286Z [info ] 2023-09-26 03:37:42,921 | INFO | tap-shopify | Skipping deselected stream 'company_locations'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.922427Z [info ] 2023-09-26 03:37:42,921 | INFO | tap-shopify | Skipping deselected stream 'customers'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.922583Z [info ] 2023-09-26 03:37:42,921 | INFO | tap-shopify | Skipping deselected stream 'deletion_events'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.922813Z [info ] 2023-09-26 03:37:42,921 | INFO | tap-shopify | Skipping deselected stream 'delivery_customizations'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.922969Z [info ] 2023-09-26 03:37:42,921 | INFO | tap-shopify | Skipping deselected stream 'discount_nodes'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.923121Z [info ] 2023-09-26 03:37:42,922 | INFO | tap-shopify | Skipping deselected stream 'discount_redeem_code_saved_searches'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.923297Z [info ] 2023-09-26 03:37:42,922 | INFO | tap-shopify | Skipping deselected stream 'draft_orders'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.923425Z [info ] 2023-09-26 03:37:42,922 | INFO | tap-shopify | Skipping deselected stream 'files'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.923546Z [info ] 2023-09-26 03:37:42,922 | INFO | tap-shopify | Skipping deselected stream 'fulfillment_orders'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.923743Z [info ] 2023-09-26 03:37:42,922 | INFO | tap-shopify | Skipping deselected stream 'gift_cards'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.924104Z [info ] 2023-09-26 03:37:42,922 | INFO | tap-shopify | Skipping deselected stream 'inventory_items'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.924245Z [info ] 2023-09-26 03:37:42,923 | INFO | tap-shopify | Skipping deselected stream 'locations'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.924715Z [info ] 2023-09-26 03:37:42,923 | INFO | tap-shopify | Skipping deselected stream 'manual_holds_fulfillment_orders'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.924888Z [info ] 2023-09-26 03:37:42,923 | INFO | tap-shopify | Skipping deselected stream 'marketing_activities'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.925125Z [info ] 2023-09-26 03:37:42,923 | INFO | tap-shopify | Skipping deselected stream 'marketing_events'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.925266Z [info ] 2023-09-26 03:37:42,923 | INFO | tap-shopify | Skipping deselected stream 'metafield_definitions'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.925401Z [info ] 2023-09-26 03:37:42,924 | INFO | tap-shopify | Skipping deselected stream 'metaobjects'. cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.926801Z [info ] 2023-09-26 03:37:42,926 | INFO | tap-shopify | Beginning incremental sync of 'orders'... cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:42.927054Z [info ] 2023-09-26 03:37:42,926 | INFO | tap-shopify | Tap has custom mapper. Using 1 provided map(s). cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.345970Z [info ] 2023-09-26 03:37:43,345 | INFO | target-postgres | Initializing 'target-postgres' target sink... cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.747843Z [info ] 2023-09-26 03:37:43,747 | INFO | singer_sdk.metrics | METRIC: {"type": "counter", "metric": "http_request_count", "value": 0, "tags": {"stream": "orders", "endpoint": ""}} cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.748230Z [info ] 2023-09-26 03:37:43,747 | INFO | singer_sdk.metrics | METRIC: {"type": "timer", "metric": "sync_duration", "value": 0.43436479568481445, "tags": {"stream": "orders", "context": {}, "status": "failed"}} cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.748782Z [info ] 2023-09-26 03:37:43,747 | INFO | singer_sdk.metrics | METRIC: {"type": "counter", "metric": "record_count", "value": 0, "tags": {"stream": "orders", "context": {}}} cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.749224Z [info ] 2023-09-26 03:37:43,747 | ERROR | tap-shopify | An unhandled error occurred while syncing 'orders' cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.749431Z [info ] Traceback (most recent call last): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.749674Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1175, in sync cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.749841Z [info ] for _ in self._sync_records(context=context): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.750045Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1070, in _sync_records cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.750237Z [info ] for record_result in self.get_records(current_context): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.750425Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 576, in get_records cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.750592Z [info ] for record in self.request_records(context): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.750713Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 393, in request_records cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.750873Z [info ] prepared_request = self.prepare_request( cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.751072Z [info ] ^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.751211Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 364, in prepare_request cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.751331Z [info ] request_data = self.prepare_request_payload(context, next_page_token) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.751481Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.751577Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 365, in prepare_request_payload cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.751659Z [info ] query = self.query.lstrip() cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.752237Z [info ] ^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.752333Z [info ] File "/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py", line 1001, in __get__ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.752488Z [info ] val = self.func(instance) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.753216Z [info ] ^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.753334Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 497, in query cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.753447Z [info ] self.evaluate_query() cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.753571Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 521, in evaluate_query cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.753771Z [info ] error_code = error.get("extensions", {}).get("code") cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754028Z [info ] ^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754275Z [info ] AttributeError: 'str' object has no attribute 'get' cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754408Z [info ] Traceback (most recent call last): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754525Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/bin/tap-shopify", line 8, in <module> cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754682Z [info ] sys.exit(TapShopify.cli()) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754773Z [info ] ^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754858Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.754953Z [info ] return self.main(*args, **kwargs) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755077Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755185Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755277Z [info ] rv = self.invoke(ctx) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755363Z [info ] ^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755448Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755539Z [info ] return ctx.invoke(self.callback, **ctx.params) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755627Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755725Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755815Z [info ] return __callback(*args, **kwargs) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755900Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.755985Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/tap_base.py", line 498, in invoke cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756071Z [info ] tap.sync_all() cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756155Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/tap_base.py", line 456, in sync_all cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756240Z [info ] stream.sync() cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756325Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1182, in sync cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756413Z [info ] raise ex cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756497Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1175, in sync cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756625Z [info ] for _ in self._sync_records(context=context): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756711Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/core.py", line 1070, in _sync_records cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756798Z [info ] for record_result in self.get_records(current_context): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756884Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 576, in get_records cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.756983Z [info ] for record in self.request_records(context): cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757069Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 393, in request_records cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757157Z [info ] prepared_request = self.prepare_request( cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757242Z [info ] ^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757326Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/singer_sdk/streams/rest.py", line 364, in prepare_request cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757413Z [info ] request_data = self.prepare_request_payload(context, next_page_token) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757497Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757583Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 365, in prepare_request_payload cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757668Z [info ] query = self.query.lstrip() cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.757751Z [info ] ^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.758102Z [info ] File "/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py", line 1001, in __get__ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.758209Z [info ] val = self.func(instance) cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.758300Z [info ] ^^^^^^^^^^^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.759214Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 497, in query cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.759335Z [info ] self.evaluate_query() cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.759432Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/extractors/tap-shopify/venv/lib/python3.11/site-packages/tap_shopify/client.py", line 521, in evaluate_query cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.759530Z [info ] error_code = error.get("extensions", {}).get("code") cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.759619Z [info ] ^^^^^^^^^ cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.759705Z [info ] AttributeError: 'str' object has no attribute 'get' cmd_type=extractor name=tap-shopify run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:43.824886Z [info ] 2023-09-26 03:37:43,824 | INFO | paramiko.transport | Connected (version 2.0, client OpenSSH_8.9p1) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:44.147057Z [info ] 2023-09-26 03:37:44,146 | INFO | paramiko.transport | Authentication (publickey) successful! cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:37:44.147782Z [info ] 2023-09-26 03:37:44,147 | INFO | target-postgres | Initializing target sink for stream 'orders'... cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.093174Z [info ] Traceback (most recent call last): cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.093428Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/bin/target-postgres", line 8, in <module> cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.094068Z [info ] sys.exit(TargetPostgres.cli()) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.094268Z [info ] ^^^^^^^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.094420Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.095533Z [info ] return self.main(*args, **kwargs) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.095687Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.095921Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.096179Z [info ] rv = self.invoke(ctx) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.096326Z [info ] ^^^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.096459Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.096618Z [info ] return ctx.invoke(self.callback, **ctx.params) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.096825Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.096960Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.097092Z [info ] return __callback(*args, **kwargs) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.097218Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.097343Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 553, in invoke cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.097556Z [info ] target.listen(file_input) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.097693Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/io_base.py", line 34, in listen cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.097958Z [info ] self._process_lines(file_input) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.098134Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 294, in _process_lines cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.098407Z [info ] counter = super()._process_lines(file_input) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.098563Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.098701Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/io_base.py", line 90, in _process_lines cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.098842Z [info ] self._process_schema_message(line_dict) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.098982Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 406, in _process_schema_message cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.099117Z [info ] _ = self.get_sink( cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.099250Z [info ] ^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.099383Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 769, in get_sink cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.099612Z [info ] return self.add_sqlsink(stream_name, schema, key_properties) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.099757Z [info ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.099890Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 703, in add_sqlsink cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.100023Z [info ] sink.setup() cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.100153Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/target_postgres/sinks.py", line 49, in setup cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.100311Z [info ] self.connector.prepare_table( cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.100440Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/target_postgres/connector.py", line 117, in prepare_table cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.100572Z [info ] self.prepare_column( cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.100874Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/target_postgres/connector.py", line 378, in prepare_column cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.101136Z [info ] self._adapt_column_type( cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.101789Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/target_postgres/connector.py", line 500, in _adapt_column_type cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.101950Z [info ] raise NotImplementedError(msg) cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.102089Z [info ] NotImplementedError: Altering columns is not supported. Could not convert column 'meltano_tap_shopify.orders.customer__id' from 'BIGINT' to 'VARCHAR'. cmd_type=loader name=target-postgres run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.280321Z [error ] Extraction failed code=1 message=AttributeError: 'str' object has no attribute 'get' name=meltano run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres
2023-09-26T08:39:07.280560Z [error ] Loading failed code=1 message=NotImplementedError: Altering columns is not supported. Could not convert column 'meltano_tap_shopify.orders.customer__id' from 'BIGINT' to 'VARCHAR'. name=meltano run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres
2023-09-26T08:39:07.281078Z [info ] ELT could not be completed: Extractor and loader failed. cmd_type=elt name=meltano run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.295006Z [info ] For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag. cmd_type=elt name=meltano run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.295169Z [info ] Note that you can also check the generated log file at '/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/logs/elt/2023-09-26T083712--tap-shopify--target-postgres/445e675a-a305-4742-9599-0584ca67a4a3/elt.log'. cmd_type=elt name=meltano run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr
2023-09-26T08:39:07.295315Z [info ] For more information on debugging and logging: https://docs.meltano.com/reference/command-line-interface#debugging cmd_type=elt name=meltano run_id=445e675a-a305-4742-9599-0584ca67a4a3 state_id=2023-09-26T083712--tap-shopify--target-postgres stdio=stderr |
Hi @konung, I am currently working on refactoring the main logic of this tap, there are a lot of constrains with the GraphQL API, and the logs are not great. This erros usually happens when the query is too long, try to select just a few fields to test and see if it woks, if not, I can try to help you to debug the error. I hope I have the refactoring of the tap done in a week or two, but feedback on this implementation is welcome. |
@sehnem Thank you so much for looking into this. I tried to transfer a bare minimum of attributes for an order and still hit errors. Also if I'm reading Meltano docs correctly - even if I select only 2 attributes - the ones that are marked as "automatic" will still be selected even for those entities that I don't want to transfer ? Here is my meltano.yml version: 1
default_environment: dev
project_id: 5ce76c58-daa5-410c-9e9f-3893d658b5cc
environments:
- name: dev
- name: staging
- name: prod
plugins:
extractors:
- name: tap-spreadsheets-anywhere
variant: ets
pip_url: git+https://github.com/ets/tap-spreadsheets-anywhere.git
config:
tables:
- path: https://ip-ranges.amazonaws.com
format: json
start_date: '2022-12-01T00:00:00Z'
key_properties: [ip_prefix]
name: aws_ips
pattern: ip-ranges.json
json_path: prefixes
- name: tap-shopify
variant: sehnem
pip_url: git+https://github.com/sehnem/tap-shopify.git
config:
api_version: 2023-10
bulk: false
flattening_enabled: true
flattening_max_depth: 3
start_date: '2023-09-25'
use_numeric_ids: false
select:
- orders.id
- orders.legacyResourceId
- orders.createdAt
- orders.email
- orders.updatedAt
catalog: extract/shopify.sehnem.catalog.json
loaders:
- name: target-postgres
variant: meltanolabs
pip_url: meltanolabs-target-postgres~=0.0.7
- name: target-jsonl
variant: andyh1203
pip_url: target-jsonl
config:
do_timestamp_file: true
custom_name: slshopify
- name: target-csv
variant: hotgluexyz
pip_url: git+https://github.com/hotgluexyz/target-csv.git
utilities:
- name: airflow
variant: apache
pip_url: git+https://github.com/meltano/airflow-ext.git@main apache-airflow==2.3.3
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.3.3/constraints-no-providers-${MELTANO__PYTHON_VERSION}.txt
When I try to run
2023-09-26T15:42:13.361009Z [info ] Running extract & load... name=meltano run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl
2023-09-26T15:42:13.961573Z [warning ] No state was found, complete import.
2023-09-26T15:42:13.966160Z [info ] Found catalog in /Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/extract/shopify.sehnem.catalog.json
2023-09-26T15:42:27.186884Z [info ] 2023-09-26 10:42:27,186 | INFO | tap-shopify | Beginning full_table sync of 'automatic_discount_nodes'... cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.188026Z [info ] 2023-09-26 10:42:27,186 | INFO | tap-shopify | Tap has custom mapper. Using 1 provided map(s). cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.567558Z [info ] 2023-09-26 10:42:27,567 | INFO | singer_sdk.metrics | METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.159894, "tags": {"stream": "automatic_discount_nodes", "endpoint": "", "http_status_code": 200, "status": "succeeded"}} cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.585098Z [info ] 2023-09-26 10:42:27,584 | WARNING | tap-shopify | Properties ('edges', 'pageInfo') were present in the 'automatic_discount_nodes' stream but not found in catalog schema. Ignoring. cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.585424Z [info ] 2023-09-26 10:42:27,585 | INFO | singer_sdk.metrics | METRIC: {"type": "counter", "metric": "http_request_count", "value": 1, "tags": {"stream": "automatic_discount_nodes", "endpoint": ""}} cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.585578Z [info ] 2023-09-26 10:42:27,585 | INFO | singer_sdk.metrics | METRIC: {"type": "timer", "metric": "sync_duration", "value": 0.39823007583618164, "tags": {"stream": "automatic_discount_nodes", "context": {}, "status": "succeeded"}} cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.585707Z [info ] 2023-09-26 10:42:27,585 | INFO | singer_sdk.metrics | METRIC: {"type": "counter", "metric": "record_count", "value": 1, "tags": {"stream": "automatic_discount_nodes", "context": {}}} cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.585930Z [info ] 2023-09-26 10:42:27,585 | INFO | tap-shopify | Beginning full_table sync of 'catalogs'... cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.586061Z [info ] 2023-09-26 10:42:27,585 | INFO | tap-shopify | Tap has custom mapper. Using 1 provided map(s). cmd_type=extractor name=tap-shopify run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.586490Z [info ] Traceback (most recent call last): cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.586644Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-jsonl/venv/bin/target-jsonl", line 8, in <module> cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.621001Z [info ] sys.exit(main()) cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.621178Z [info ] ^^^^^^ cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.621321Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-jsonl/venv/lib/python3.11/site-packages/target_jsonl.py", line 92, in main cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.621476Z [info ] state = persist_messages( cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.621609Z [info ] ^^^^^^^^^^^^^^^^^ cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.621737Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-jsonl/venv/lib/python3.11/site-packages/target_jsonl.py", line 54, in persist_messages cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.621876Z [info ] validators[o['stream']].validate((o['record'])) cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.622006Z [info ] File "/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/loaders/target-jsonl/venv/lib/python3.11/site-packages/jsonschema/validators.py", line 130, in validate cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.622135Z [info ] raise error cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.622263Z [info ] jsonschema.exceptions.ValidationError: 'id' is a required property cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.622392Z [info ] cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.622519Z [info ] Failed validating 'required' in schema: cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.622713Z [info ] {'properties': {'id': {'type': ['string']}, cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.622938Z [info ] 'metafieldDefinitions__edges': {'items': {'properties': {'node': {'properties': {'id': {'type': ['string']}}, cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.623097Z [info ] 'required': ['id'], cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.623239Z [info ] 'type': 'object'}}, cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.623390Z [info ] 'required': ['node'], cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.623521Z [info ] 'type': 'object'}, cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.623863Z [info ] 'type': 'array'}}, cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.623999Z [info ] 'required': ['id', 'metafieldDefinitions'], cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.624128Z [info ] 'type': 'object'} cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.624276Z [info ] cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.624465Z [info ] On instance: cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.624618Z [info ] {} cmd_type=loader name=target-jsonl run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.651955Z [error ] Loading failed code=1 message= {} name=meltano run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl
2023-09-26T15:42:27.652464Z [info ] ELT could not be completed: Loader failed. cmd_type=elt name=meltano run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.652624Z [info ] For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag. cmd_type=elt name=meltano run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.652763Z [info ] Note that you can also check the generated log file at '/Users/konung/Projects/konungshop/Code/konungshop-elt-meltano/konungshop-elt/.meltano/logs/elt/2023-09-26T154212--tap-shopify--target-jsonl/7f0da754-0c3f-480d-b780-2328ed721108/elt.log'. cmd_type=elt name=meltano run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr
2023-09-26T15:42:27.652903Z [info ] For more information on debugging and logging: https://docs.meltano.com/reference/command-line-interface#debugging cmd_type=elt name=meltano run_id=7f0da754-0c3f-480d-b780-2328ed721108 state_id=2023-09-26T154212--tap-shopify--target-jsonl stdio=stderr |
This is a known error, use bulk as true and it should be solved. The current version of the tap is not in a good shape, I am fixing a lot of things in the structure right now and things may not be working as they should. In the next week I will release a new version with fixes for these cases. |
Thanks @sehnem . I tried bulk per your suggestion, but still ran into more issues with schema and empty fields. I will just wait till your next release to test again. For now I'm using airbyte, even if it is painfully slow :D It does the job, thou, I was just looking for an option to speed up things |
The bulk API is much faster, but it is much more complex with a lot of
edgecases, I will ping you once I release the new version.
Em qui., 28 de set. de 2023 às 01:50, Nick Gorbikoff <
***@***.***> escreveu:
… Thanks @sehnem <https://github.com/sehnem> . I tried bulk per your
suggestion, but still ran into more issues with schema and empty fields. I
will just wait till your next release to test again. For now I'm using
airbyte, even if it is painfully slow :D It does the job, thou.
—
Reply to this email directly, view it on GitHub
<#9 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAOUJ3VHQ3KORDYKFFF6QJ3X4T6ZTANCNFSM6AAAAAA5HJJFQ4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Excellent. Thank you!!
Have a great day!
Nick
Message ID: ***@***.***>
|
Hi .
I've been testing meltano ( a total meltano noob here) with this tap-shopify variant, and keep hitting all kinds of errors when trying to run actual el/run .
I know it's not a credentials issue:
tap-spreadsheets-anywhere
totarget-postgres
for other data.meltano config tap-shopify test
runs greenmeltano invoke tap-shopify --discover > ./shopify_catalog.json
meltano invoke tap-shopify --catalog=./shopify_catalog.json
- I can see actual live data pulled form my store.I seem to be missing some crucial step in tap-shopify setup, that's probably obvious to a more experienced Meltano user.
For example: do I need to somehow point my
meltano el tap-shopify target-postgres
to use--catalog=shopify_catalog.json
or possibly do something else?The text was updated successfully, but these errors were encountered: