🎉 New Source: NetSuite #16093

drrest · 2022-08-29T14:56:24Z

What

New Source NetSuite based on NetSuite REST API

🚨 User Impact 🚨

No impact is expected.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

drrest · 2022-08-29T14:57:26Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2949527182
❌ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2949527182
🐛 https://gradle.com/s/7jgag6ndf6sdm

Build Failed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestIncremental.test_two_sequential_reads because not found in the config
============================== 1 skipped in 0.76s ==============================

airbyte-integrations/connectors/source-netsuite/acceptance-test-config.yml

airbyte-integrations/connectors/source-netsuite/integration_tests/catalog.json

airbyte-integrations/connectors/source-netsuite/integration_tests/configured_catalog.json

airbyte-integrations/connectors/source-netsuite/source_netsuite/schemas/accounts.json

airbyte-integrations/connectors/source-netsuite/source_netsuite/streams.py

changed: several fixes

drrest · 2022-09-01T13:15:17Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2971925632
❌ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2971925632
🐛 https://gradle.com/s/xlsfyis336qtg

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - docker.errors.Cont...
FAILED test_core.py::TestConnection::test_check[inputs1] - docker.errors.Cont...
ERROR test_core.py::TestSpec::test_config_match_spec[inputs0] - FileNotFoundE...
ERROR test_core.py::TestDiscovery::test_discover[inputs0] - FileNotFoundError...
ERROR test_core.py::TestDiscovery::test_defined_cursors_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_refs_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-allOf]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-not]
ERROR test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_streams_has_sync_modes[inputs0] - Fil...
ERROR test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
ERROR test_core.py::TestBasicRead::test_read[inputs0] - FileNotFoundError: [E...
ERROR test_core.py::TestBasicRead::test_airbyte_trace_message_on_failure[inputs0]
ERROR test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestIncremental.test_two_sequential_reads because not found in the config
SKIPPED [5] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:55: The previous connector image could not be retrieved.
============= 2 failed, 10 passed, 6 skipped, 12 errors in 12.07s ==============

drrest · 2022-09-01T13:46:50Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2972154329
❌ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2972154329
🐛 https://gradle.com/s/itmjxmypglnuu

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - docker.errors.Cont...
FAILED test_core.py::TestConnection::test_check[inputs1] - docker.errors.Cont...
FAILED test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0]
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AttributeError: 'lis...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestIncremental.test_two_sequential_reads because not found in the config
SKIPPED [5] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:55: The previous connector image could not be retrieved.
============= 4 failed, 20 passed, 6 skipped in 111.71s (0:01:51) ==============

drrest · 2022-09-01T14:16:28Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2972380253
❌ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2972380253
🐛 https://gradle.com/s/aslbwe2lpsgjs

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestConnection::test_check[inputs1] - docker.errors.Cont...
FAILED test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0]
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AttributeError: 'lis...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestIncremental.test_two_sequential_reads because not found in the config
SKIPPED [5] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:55: The previous connector image could not be retrieved.
============= 4 failed, 20 passed, 6 skipped in 121.72s (0:02:01) ==============

drrest · 2022-09-05T22:16:00Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2996297632
❌ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2996297632
🐛 https://gradle.com/s/pbcpjxzz3giyi

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestConnection::test_check[inputs1] - docker.errors.Cont...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestIncremental.test_two_sequential_reads because not found in the config
SKIPPED [5] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:55: The previous connector image could not be retrieved.
============= 2 failed, 22 passed, 6 skipped in 162.51s (0:02:42) ==============

drrest · 2022-09-06T10:56:33Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2999588182
✅ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/2999588182
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        82      6    93%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/tests/test_incremental.py       121     25    79%
source_acceptance_test/utils/common.py                  77     17    78%
source_acceptance_test/tests/test_core.py              355    107    70%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
------------------------------------------------------------------------
TOTAL                                                 1023    247    76%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestIncremental.test_two_sequential_reads because not found in the config
SKIPPED [5] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:55: The previous connector image could not be retrieved.
================== 24 passed, 6 skipped in 107.38s (0:01:47) ===================

drrest · 2022-09-06T16:36:19Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3001866788
❌ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3001866788
🐛 https://gradle.com/s/aggcdpe45nyro

Build Failed

Test summary info:

Could not find result summary

davydov-d

Good, I believe we will resolve those unresolved comments during the certification to beta

lazebnyi · 2022-09-15T19:04:03Z

Good, I believe we will resolve those unresolved comments during the certification to beta

@bazarnov Let's create issue to not forget about that. Thanks for great work!

bazarnov · 2022-09-16T00:30:40Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3064628720
❌ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3064628720
🐛 https://gradle.com/s/avmhqoocoaki2

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:51: The previous connector image could not be retrieved.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:229: The previous connector image could not be retrieved.
============= 1 failed, 26 passed, 2 skipped in 4831.89s (1:20:31) =============

bazarnov · 2022-09-16T09:22:04Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3066781932
✅ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3066781932
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       121     25    79%   21-23, 29-31, 36-43, 48-61, 208-216
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       110     48    56%   23-26, 32, 36, 39-64, 67-69, 72-74, 77-79, 82-84, 87-89, 92-110, 144-146
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1325    463    65%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:51: The previous connector image could not be retrieved.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:229: The previous connector image could not be retrieved.
================== 27 passed, 2 skipped in 2429.57s (0:40:29) ==================

…e-new-connector

bazarnov · 2022-09-18T13:05:51Z

/test connector=connectors/source-netsuite

🕑 connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3077241421
✅ connectors/source-netsuite https://github.com/airbytehq/airbyte/actions/runs/3077241421
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       121     25    79%   21-23, 29-31, 36-43, 48-61, 208-216
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       110     48    56%   23-26, 32, 36, 39-64, 67-69, 72-74, 77-79, 82-84, 87-89, 92-110, 144-146
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1325    463    65%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:51: The previous connector image could not be retrieved.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:229: The previous connector image could not be retrieved.
================== 27 passed, 2 skipped in 2661.31s (0:44:21) ==================

bazarnov · 2022-09-21T17:24:52Z

/publish connector=connectors/source-netsuite

🕑 Publishing the following connectors:
connectors/source-netsuite
https://github.com/airbytehq/airbyte/actions/runs/3099821149

Connector	Did it publish?	Were definitions generated?
connectors/source-netsuite	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

Source Netsuite - base realization with full_refresh only points

2b51448

octavia-squidington-iii added the connectors/source/netsuite label Aug 29, 2022

github-actions bot added the area/connectors Connector related issues label Aug 29, 2022

drrest requested a review from lazebnyi August 29, 2022 14:57

Source Netsuite - changed acceptanceTest settings

c2e3bec

This was linked to issues Aug 29, 2022

New source: Oracle Netsuite Analytics #3212

Closed

[EPIC] Source Oracle Netsuite to Alpha #13349

Closed

drrest removed a link to an issue Aug 29, 2022

[EPIC] Source Oracle Netsuite to Alpha #13349

Closed

11 tasks

lazebnyi requested review from bazarnov, grubberr, roman-yermilov-gl, davydov-d and arsenlosenko August 31, 2022 15:05

davydov-d requested changes Sep 1, 2022

View reviewed changes

Source Netsuite:

7039a8e

changed: several fixes

little fixes

ea6ccc9

drrest added 2 commits September 2, 2022 22:09

Source Netsuite - fixes to schemas

5b3f4bc

Source Netsuite - base realization with full_refresh only points

97fc8ba

drrest added 3 commits September 6, 2022 13:03

Source Netsuite

1fac4cb

Source Netsuite - acceptanceTest fix

be2f6a8

Source Netsuite

a6d55dd

Source Netsuite - changes

b4060be

Source Netsuite

2a0eac8

davydov-d approved these changes Sep 15, 2022

View reviewed changes

bazarnov approved these changes Sep 15, 2022

View reviewed changes

bazarnov added 2 commits September 16, 2022 00:13

fixed build.md

5507542

updated and formated code

cc6b9dd

bazarnov temporarily deployed to more-secrets September 16, 2022 00:31 Inactive

updated cursor reference

eeab96b

bazarnov temporarily deployed to more-secrets September 16, 2022 00:37 Inactive

updated request params

5865198

bazarnov temporarily deployed to more-secrets September 16, 2022 09:24 Inactive

Merge remote-tracking branch 'origin/master' into 3212-source-netsuit…

9a85678

…e-new-connector

bazarnov temporarily deployed to more-secrets September 16, 2022 11:26 Inactive

arsenlosenko approved these changes Sep 19, 2022

View reviewed changes

auto-bump connector version [ci skip]

6008766

octavia-squidington-iii added the bounty label Sep 21, 2022

bazarnov merged commit db86fcb into master Sep 21, 2022

bazarnov deleted the 3212-source-netsuite-new-connector branch September 21, 2022 18:43

swatosh mentioned this pull request Sep 21, 2022

🎉 New Source: Netsuite #13086

Closed

19 tasks

octavia-squidington-iii mentioned this pull request Sep 21, 2022

Bump Airbyte version from 0.40.7 to 0.40.8 #17009

Merged

sajarin added internal and removed bounty labels Sep 22, 2022

robbinhan pushed a commit to robbinhan/airbyte that referenced this pull request Sep 29, 2022

🎉 New Source: NetSuite (airbytehq#16093)

d93a2d3

natalyjazzviolin mentioned this pull request Oct 18, 2022

🐛 Source Netsuite: Determine request date format #17470

Closed

14 tasks

jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022

🎉 New Source: NetSuite (airbytehq#16093)

1f0beeb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 New Source: NetSuite #16093

🎉 New Source: NetSuite #16093

drrest commented Aug 29, 2022 •

edited by sherifnada

Loading

drrest commented Aug 29, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 1, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 1, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 1, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 5, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 6, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 6, 2022 •

edited by github-actions bot

Loading

davydov-d left a comment

lazebnyi commented Sep 15, 2022

bazarnov commented Sep 16, 2022 •

edited by github-actions bot

Loading

bazarnov commented Sep 16, 2022 •

edited by github-actions bot

Loading

bazarnov commented Sep 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Sep 21, 2022 •

edited by github-actions bot

Loading

🎉 New Source: NetSuite #16093

🎉 New Source: NetSuite #16093

Conversation

drrest commented Aug 29, 2022 • edited by sherifnada Loading

What

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

drrest commented Aug 29, 2022 • edited by github-actions bot Loading

Build Failed

drrest commented Sep 1, 2022 • edited by github-actions bot Loading

Build Failed

drrest commented Sep 1, 2022 • edited by github-actions bot Loading

Build Failed

drrest commented Sep 1, 2022 • edited by github-actions bot Loading

Build Failed

drrest commented Sep 5, 2022 • edited by github-actions bot Loading

Build Failed

drrest commented Sep 6, 2022 • edited by github-actions bot Loading

Build Passed

drrest commented Sep 6, 2022 • edited by github-actions bot Loading

Build Failed

davydov-d left a comment

Choose a reason for hiding this comment

lazebnyi commented Sep 15, 2022

bazarnov commented Sep 16, 2022 • edited by github-actions bot Loading

Build Failed

bazarnov commented Sep 16, 2022 • edited by github-actions bot Loading

Build Passed

bazarnov commented Sep 18, 2022 • edited by github-actions bot Loading

Build Passed

bazarnov commented Sep 21, 2022 • edited by github-actions bot Loading

drrest commented Aug 29, 2022 •

edited by sherifnada

Loading

drrest commented Aug 29, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 1, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 1, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 1, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 5, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 6, 2022 •

edited by github-actions bot

Loading

drrest commented Sep 6, 2022 •

edited by github-actions bot

Loading

bazarnov commented Sep 16, 2022 •

edited by github-actions bot

Loading

bazarnov commented Sep 16, 2022 •

edited by github-actions bot

Loading

bazarnov commented Sep 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Sep 21, 2022 •

edited by github-actions bot

Loading