Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GitHub Native Connector: max retries hit, terminates #5963

Closed
garden-of-delete opened this issue Sep 10, 2021 · 1 comment · Fixed by #9999
Closed

GitHub Native Connector: max retries hit, terminates #5963

garden-of-delete opened this issue Sep 10, 2021 · 1 comment · Fixed by #9999
Assignees
@annalvova05
Labels
area/connectors Connector related issues connectors/source/github connectors/sources-api type/bug Something isn't working

Comments

@garden-of-delete
Copy link

garden-of-delete commented Sep 10, 2021 •
edited

Enviroment

  • Airbyte version: example is 0.29.13-alpha
  • OS Version / Instance: AWS EC2
  • Deployment: Docker
  • Source Connector and version: GitHub Native v0.1.10
  • Destination Connector and version: BigQuery v0.1.1
  • Severity: High
  • Step where error happened: Sync Job

Current Behavior

Sync job fails, seemingly as a function of repo size + number of active sync jobs, which seems to implicate the GitHub API rate limit.

I suspect this is an issue with hitting the rate limit on the GitHub API. Logging output that looks like...

2021-09-10 03:33:31 ERROR () DefaultAirbyteStreamFactory(internalLog):108 - Max retry limit reached. Request: <PreparedRequest [GET]>, Response: <Response [502]>
2021-09-10 03:33:31 ERROR () DefaultAirbyteStreamFactory(internalLog):108 - Undefined error while reading records: 
2021-09-10 03:33:31 ERROR () DefaultAirbyteStreamFactory(internalLog):108 - Encountered an exception while reading stream SourceGithub
Traceback (most recent call last): ...

Seems to accompany the source worker finishing in all the logs I have looked through.

Expected Behavior

One would expect the sync operation to finish while handling pauses from hitting the API rate limit.

Logs

LOG 
2021-09-10 03:32:20 INFO () WorkerRun(call):62 - Executing worker wrapper. Airbyte version: 0.29.13-alpha
2021-09-10 03:32:20 INFO () TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.29.13-alpha
2021-09-10 03:32:20 INFO () DefaultReplicationWorker(run):102 - start sync worker. job id: 1528 attempt id: 0
2021-09-10 03:32:20 INFO () DefaultReplicationWorker(run):111 - configured sync modes: {null.commit_comments=full_refresh - append, null.assignees=full_refresh - append, null.issue_events=full_refresh - append, null.issues=full_refresh - append, null.issue_milestones=full_refresh - append, null.reviews=full_refresh - append, null.issue_comment_reactions=full_refresh - append, null.issue_reactions=full_refresh - append, null.commit_comment_reactions=full_refresh - append, null.tags=full_refresh - append, null.pull_request_comment_reactions=full_refresh - append, null.pull_requests=full_refresh - append, null.comments=full_refresh - append, null.commits=full_refresh - append, null.issue_labels=full_refresh - append, null.organizations=full_refresh - append, null.branches=full_refresh - append, null.review_comments=full_refresh - append, null.pull_request_stats=full_refresh - append, null.releases=full_refresh - append, null.projects=full_refresh - append, null.events=full_refresh - append, null.users=full_refresh - append, null.repositories=full_refresh - append, null.stargazers=full_refresh - append}
2021-09-10 03:32:20 INFO () DefaultAirbyteDestination(start):78 - Running destination...
2021-09-10 03:32:20 INFO () LineGobbler(voidCall):85 - Checking if airbyte/destination-bigquery:0.4.0 exists...
2021-09-10 03:32:20 INFO () LineGobbler(voidCall):85 - airbyte/destination-bigquery:0.4.0 was found locally.
2021-09-10 03:32:20 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1528/0 --network host --log-driver none airbyte/destination-bigquery:0.4.0 write --config destination_config.json --catalog destination_catalog.json
2021-09-10 03:32:20 INFO () LineGobbler(voidCall):85 - Checking if airbyte/source-github:0.1.10 exists...
2021-09-10 03:32:20 INFO () LineGobbler(voidCall):85 - airbyte/source-github:0.1.10 was found locally.
2021-09-10 03:32:20 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1528/0 --network host --log-driver none airbyte/source-github:0.1.10 read --config source_config.json --catalog source_catalog.json --state input_state.json
2021-09-10 03:32:20 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):246 - Destination output thread started.
2021-09-10 03:32:20 INFO () DefaultReplicationWorker(run):139 - Waiting for source thread to join.
2021-09-10 03:32:20 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):210 - Replication thread started.
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Starting syncing SourceGithub
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Syncing stream: assignees 
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Read 61 records from assignees stream
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Syncing stream: branches 
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:22 �[32mINFO�[m i.a.i.d.b.BigQueryDestination(main):356 - {} - starting destination: class io.airbyte.integrations.destination.bigquery.BigQueryDestination
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:22 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:22 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:22 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):100 - {} - Command: WRITE
2021-09-10 03:32:22 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:22 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2021-09-10 03:32:23 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:23 �[33mWARN�[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-10 03:32:23 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:23 �[33mWARN�[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-10 03:32:23 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:23 �[32mINFO�[m i.a.i.d.b.BigQueryDestination(getLoadingMethod):331 - {} - Selected loading method is set to: STANDARD
2021-09-10 03:32:23 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:23 �[32mINFO�[m i.a.i.d.b.BigQueryDestination(isKeepFilesInGcs):344 - {} - All tmp files will be removed from GCS when migration is finished
2021-09-10 03:32:24 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:24 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_xpz_github_superset_assignees}} created successfully
2021-09-10 03:32:25 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:25 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_alu_github_superset_branches}} created successfully
2021-09-10 03:32:25 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:25 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_akt_github_superset_comments}} created successfully
2021-09-10 03:32:26 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:26 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_vhv_github_superset_commit_comment_reactions}} created successfully
2021-09-10 03:32:26 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:26 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_gxa_github_superset_commit_comments}} created successfully
2021-09-10 03:32:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:27 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_eaw_github_superset_commits}} created successfully
2021-09-10 03:32:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:27 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_msm_github_superset_events}} created successfully
2021-09-10 03:32:28 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:28 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_hzs_github_superset_issue_comment_reactions}} created successfully
2021-09-10 03:32:28 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:28 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_cca_github_superset_issue_events}} created successfully
2021-09-10 03:32:29 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:29 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_igi_github_superset_issue_labels}} created successfully
2021-09-10 03:32:29 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:29 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_wmf_github_superset_issue_milestones}} created successfully
2021-09-10 03:32:30 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:30 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_imi_github_superset_issue_reactions}} created successfully
2021-09-10 03:32:30 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:30 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_kvw_github_superset_issues}} created successfully
2021-09-10 03:32:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:31 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_tux_github_superset_organizations}} created successfully
2021-09-10 03:32:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:31 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_ikp_github_superset_projects}} created successfully
2021-09-10 03:32:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:31 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_mny_github_superset_pull_request_comment_reactions}} created successfully
2021-09-10 03:32:32 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:32 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_gqa_github_superset_pull_request_stats}} created successfully
2021-09-10 03:32:32 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:32 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_wim_github_superset_pull_requests}} created successfully
2021-09-10 03:32:33 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:33 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_mpe_github_superset_releases}} created successfully
2021-09-10 03:32:33 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:33 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_rmf_github_superset_repositories}} created successfully
2021-09-10 03:32:34 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:34 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_vyb_github_superset_review_comments}} created successfully
2021-09-10 03:32:34 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:34 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_dwy_github_superset_reviews}} created successfully
2021-09-10 03:32:35 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:35 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_gjq_github_superset_stargazers}} created successfully
2021-09-10 03:32:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:36 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_tyx_github_superset_tags}} created successfully
2021-09-10 03:32:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:32:36 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_wpn_github_superset_users}} created successfully
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 - Traceback (most recent call last):
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/airbyte/integration_code/main.py", line 33, in <module>
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     launch(source, sys.argv[1:])
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/entrypoint.py", line 125, in launch
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     for message in source_entrypoint.run(parsed_args):
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/entrypoint.py", line 116, in run
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     for message in generator:
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 121, in read
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     raise e
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 117, in read
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     internal_config=internal_config,
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 147, in _read_stream
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     for record in record_iterator:
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 196, in _read_full_refresh
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     for record in records:
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/airbyte/integration_code/source_github/streams.py", line 269, in read_records
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     sync_mode=sync_mode, cursor_field=cursor_field, stream_slice=stream_slice, stream_state=stream_state
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/airbyte/integration_code/source_github/streams.py", line 140, in read_records
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     raise e
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/airbyte/integration_code/source_github/streams.py", line 113, in read_records
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     yield from super().read_records(stream_slice=stream_slice, **kwargs)
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 318, in read_records
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     response = self._send_request(request, request_kwargs)
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 295, in _send_request
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     return backoff_handler(user_backoff_handler)(request, request_kwargs)
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     ret = target(*args, **kwargs)
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     ret = target(*args, **kwargs)
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -   File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 256, in _send
2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 -     raise UserDefinedBackoffException(backoff=custom_backoff_time, request=request, response=response)
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Read 362 records from branches stream
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Syncing stream: comments 
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Retrying. Sleeping for 0.5 seconds
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Retrying. Sleeping for 0.5 seconds
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Retrying. Sleeping for 0.5 seconds
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Retrying. Sleeping for 0.5 seconds
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Retrying. Sleeping for 0.5 seconds
2021-09-10 03:33:31 ERROR () DefaultAirbyteStreamFactory(internalLog):108 - Max retry limit reached. Request: <PreparedRequest [GET]>, Response: <Response [502]>
2021-09-10 03:33:31 ERROR () DefaultAirbyteStreamFactory(internalLog):108 - Undefined error while reading records: 
2021-09-10 03:33:31 ERROR () DefaultAirbyteStreamFactory(internalLog):108 - Encountered an exception while reading stream SourceGithub
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 117, in read
    internal_config=internal_config,
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 147, in _read_stream
    for record in record_iterator:
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 196, in _read_full_refresh
    for record in records:
  File "/airbyte/integration_code/source_github/streams.py", line 269, in read_records
    sync_mode=sync_mode, cursor_field=cursor_field, stream_slice=stream_slice, stream_state=stream_state
  File "/airbyte/integration_code/source_github/streams.py", line 140, in read_records
    raise e
  File "/airbyte/integration_code/source_github/streams.py", line 113, in read_records
    yield from super().read_records(stream_slice=stream_slice, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 318, in read_records
    response = self._send_request(request, request_kwargs)
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 295, in _send_request
    return backoff_handler(user_backoff_handler)(request, request_kwargs)
  File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 256, in _send
    raise UserDefinedBackoffException(backoff=custom_backoff_time, request=request, response=response)
airbyte_cdk.sources.streams.http.exceptions.UserDefinedBackoffException

2021-09-10 03:33:31 ERROR () LineGobbler(voidCall):85 - airbyte_cdk.sources.streams.http.exceptions.UserDefinedBackoffException
2021-09-10 03:33:31 INFO () DefaultReplicationWorker(run):141 - Source thread complete.
2021-09-10 03:33:31 INFO () DefaultReplicationWorker(run):142 - Waiting for destination thread to join.
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:31 �[32mINFO�[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 - {} - Airbyte message consumer: succeeded.
2021-09-10 03:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:31 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(close):163 - {} - Started closing all connections
2021-09-10 03:33:32 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:32 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(closeNormalBigqueryStreams):278 - {} - Waiting for jobs to be finished/closed
2021-09-10 03:33:35 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:35 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(closeNormalBigqueryStreams):295 - {} - Migration finished with no explicit errors. Copying data from tmp tables to permanent
2021-09-10 03:33:35 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:35 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_hzs_github_superset_issue_comment_reactions}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_issue_comment_reactions}}
2021-09-10 03:33:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:36 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_wmf_github_superset_issue_milestones}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_issue_milestones}}
2021-09-10 03:33:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:36 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_wim_github_superset_pull_requests}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_pull_requests}}
2021-09-10 03:33:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:37 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_igi_github_superset_issue_labels}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_issue_labels}}
2021-09-10 03:33:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:40 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_akt_github_superset_comments}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_comments}}
2021-09-10 03:33:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:40 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_msm_github_superset_events}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_events}}
2021-09-10 03:33:41 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:41 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_kvw_github_superset_issues}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_issues}}
2021-09-10 03:33:41 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:41 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_mny_github_superset_pull_request_comment_reactions}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_pull_request_comment_reactions}}
2021-09-10 03:33:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:42 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_rmf_github_superset_repositories}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_repositories}}
2021-09-10 03:33:42 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:42 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_dwy_github_superset_reviews}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_reviews}}
2021-09-10 03:33:43 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:43 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_vhv_github_superset_commit_comment_reactions}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_commit_comment_reactions}}
2021-09-10 03:33:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:44 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_gxa_github_superset_commit_comments}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_commit_comments}}
2021-09-10 03:33:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:44 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_tux_github_superset_organizations}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_organizations}}
2021-09-10 03:33:45 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:45 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_vyb_github_superset_review_comments}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_review_comments}}
2021-09-10 03:33:45 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:45 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_tyx_github_superset_tags}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_tags}}
2021-09-10 03:33:45 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:45 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_cca_github_superset_issue_events}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_issue_events}}
2021-09-10 03:33:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:46 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_eaw_github_superset_commits}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_commits}}
2021-09-10 03:33:47 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:47 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_wpn_github_superset_users}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_users}}
2021-09-10 03:33:48 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:48 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_imi_github_superset_issue_reactions}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_issue_reactions}}
2021-09-10 03:33:48 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:48 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_mpe_github_superset_releases}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_releases}}
2021-09-10 03:33:48 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:48 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_gjq_github_superset_stargazers}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_stargazers}}
2021-09-10 03:33:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:50 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_xpz_github_superset_assignees}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_assignees}}
2021-09-10 03:33:51 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:51 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_ikp_github_superset_projects}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_projects}}
2021-09-10 03:33:54 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:54 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_gqa_github_superset_pull_request_stats}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_pull_request_stats}}
2021-09-10 03:33:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:55 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):359 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_tmp_alu_github_superset_branches}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=base, tableId=_airbyte_raw_github_superset_branches}}
2021-09-10 03:33:55 INFO () JsonSchemaValidator(test):76 - JSON schema validation failed. 
errors: $: null found, object expected
2021-09-10 03:33:55 ERROR () DefaultAirbyteStreamFactory(lambda$create$1):83 - Validation failed: null
2021-09-10 03:33:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:55 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(closeNormalBigqueryStreams):307 - {} - Removing tmp tables...
2021-09-10 03:33:59 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:59 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(closeNormalBigqueryStreams):309 - {} - Finishing destination process...completed
2021-09-10 03:33:59 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:59 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):153 - {} - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination
2021-09-10 03:33:59 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-10 03:33:59 �[32mINFO�[m i.a.i.d.b.BigQueryDestination(main):358 - {} - completed destination: class io.airbyte.integrations.destination.bigquery.BigQueryDestination
2021-09-10 03:33:59 INFO () DefaultReplicationWorker(run):144 - Destination thread complete.
2021-09-10 03:33:59 ERROR () DefaultReplicationWorker(run):148 - Sync worker failed.
io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
	at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:135) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:146) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:52) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
2021-09-10 03:33:59 INFO () DefaultReplicationWorker(run):172 - sync summary: io.airbyte.config.ReplicationAttemptSummary@3007b743[status=failed,recordsSynced=423,bytesSynced=145206,startTime=1631244740760,endTime=1631244839896]
2021-09-10 03:33:59 INFO () DefaultReplicationWorker(run):181 - Source did not output any state messages
2021-09-10 03:33:59 WARN () DefaultReplicationWorker(run):189 - State capture: No new state, falling back on input state: io.airbyte.config.State@743888a[state={}]
2021-09-10 03:33:59 INFO () TemporalAttemptExecution(get):133 - Stopping cancellation check scheduling...
2021-09-10 03:33:59 INFO () SyncWorkflow$ReplicationActivityImpl(replicate):184 - sync summary: io.airbyte.config.StandardSyncOutput@7030e260[standardSyncSummary=io.airbyte.config.StandardSyncSummary@665cfa9c[status=failed,recordsSynced=423,bytesSynced=145206,startTime=1631244740760,endTime=1631244839896],state=io.airbyte.config.State@743888a[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@68fe87ac[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@3aa3f06[stream=io.airbyte.protocol.models.AirbyteStream@747efef1[name=github_superset_assignees,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"repository":{"type":["string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@757772f8[stream=io.airbyte.protocol.models.AirbyteStream@6928581d[name=github_superset_branches,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"name":{"type":["null","string"]},"commit":{"type":["null","object"],"properties":{"sha":{"type":["null","string"]},"url":{"type":["null","string"]}}},"protected":{"type":["null","boolean"]},"protection":{"type":["null","object"],"properties":{"required_status_checks":{"type":["null","object"],"properties":{"contexts":{"type":["null","array"],"items":[{"type":["null","string"]},{"type":["null","string"]}]},"enforcement_level":{"type":["null","string"]}}}}},"repository":{"type":["string"]},"protection_url":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@3b281831[stream=io.airbyte.protocol.models.AirbyteStream@38a2f0d3[name=github_superset_comments,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"body":{"type":["null","string"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"node_id":{"type":["null","string"]},"user_id":{"type":["null","integer"]},"html_url":{"type":["null","string"]},"issue_url":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"author_association":{"type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[updated_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@768cb58d[stream=io.airbyte.protocol.models.AirbyteStream@5dcfe3f1[name=github_superset_commit_comment_reactions,jsonSchema={"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"content":{"type":["null","string"]},"node_id":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@26dd81e9[stream=io.airbyte.protocol.models.AirbyteStream@956c989[name=github_superset_commit_comments,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"body":{"type":["null","string"]},"line":{"type":["null","integer"]},"path":{"type":["null","string"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"position":{"type":["null","integer"]},"commit_id":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"author_association":{"type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[updated_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@1a6525f3[stream=io.airbyte.protocol.models.AirbyteStream@38178d25[name=github_superset_commits,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"sha":{"type":["null","string"]},"url":{"type":["null","string"]},"commit":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"tree":{"type":["null","object"],"properties":{"sha":{"type":["null","string"]},"url":{"type":["null","string"]}}},"author":{"type":["null","object"],"properties":{"date":{"type":["null","string"],"format":"date-time"},"name":{"type":["null","string"]},"email":{"type":["null","string"]}}},"message":{"type":["null","string"]},"committer":{"type":["null","object"],"properties":{"date":{"type":["null","string"],"format":"date-time"},"name":{"type":["null","string"]},"email":{"type":["null","string"]}}},"verification":{"type":["null","object"],"properties":{"reason":{"type":["null","string"]},"payload":{"type":["null","string"]},"verified":{"type":["null","boolean"]},"signature":{"type":["null","string"]}}},"comment_count":{"type":["null","integer"]}}},"node_id":{"type":["null","string"]},"parents":{"type":["null","array"],"items":{"type":["null","object"],"properties":{"sha":{"type":["null","string"]},"url":{"type":["null","string"]},"html_url":{"type":["null","string"]}}}},"html_url":{"type":["null","string"]},"author_id":{"type":["null","integer"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["string"]},"comments_url":{"type":["null","string"]},"committer_id":{"type":["null","integer"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[created_at],sourceDefinedPrimaryKey=[[sha]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[created_at],destinationSyncMode=append,primaryKey=[[sha]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@720f9107[stream=io.airbyte.protocol.models.AirbyteStream@4d2a144e[name=github_superset_events,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","string"]},"type":{"type":["null","string"]},"org_id":{"type":["null","integer"]},"public":{"type":["null","boolean"]},"payload":{"type":["null","object"],"properties":{}},"repo_id":{"type":["null","integer"]},"actor_id":{"type":["null","integer"]},"created_at":{"type":["null","string"]},"repository":{"type":["string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[created_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[created_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@7ed3faa5[stream=io.airbyte.protocol.models.AirbyteStream@7a292eff[name=github_superset_issue_comment_reactions,jsonSchema={"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"content":{"type":["null","string"]},"node_id":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@74201f14[stream=io.airbyte.protocol.models.AirbyteStream@6935fdba[name=github_superset_issue_events,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"event":{"type":["null","string"]},"node_id":{"type":["null","string"]},"actor_id":{"type":["null","integer"]},"issue_id":{"type":["null","integer"]},"commit_id":{"type":["null","string"]},"commit_url":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[created_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[created_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@8d234eb[stream=io.airbyte.protocol.models.AirbyteStream@3b3fbdd8[name=github_superset_issue_labels,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"name":{"type":["null","string"]},"color":{"type":["null","string"]},"default":{"type":["null","boolean"]},"node_id":{"type":["null","string"]},"repository":{"type":["string"]},"description":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@4f93cd14[stream=io.airbyte.protocol.models.AirbyteStream@90d2171[name=github_superset_issue_milestones,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"state":{"type":["null","string"]},"title":{"type":["null","string"]},"due_on":{"type":["null","string"],"format":"date-time"},"number":{"type":["null","integer"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"closed_at":{"type":["null","string"],"format":"date-time"},"created_at":{"type":["null","string"],"format":"date-time"},"creator_id":{"type":["null","integer"]},"labels_url":{"type":["null","string"]},"repository":{"type":["string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"description":{"type":["null","string"]},"open_issues":{"type":["null","integer"]},"closed_issues":{"type":["null","integer"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[updated_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@10e129d7[stream=io.airbyte.protocol.models.AirbyteStream@5676876e[name=github_superset_issue_reactions,jsonSchema={"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"content":{"type":["null","string"]},"node_id":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@7dc1bd9[stream=io.airbyte.protocol.models.AirbyteStream@673ab4fa[name=github_superset_issues,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"body":{"type":["null","string"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"state":{"type":["null","string"]},"title":{"type":["null","string"]},"labels":{"type":["null","array"],"items":{"type":["null","integer"]}},"locked":{"type":["null","boolean"]},"number":{"type":["null","integer"]},"node_id":{"type":["null","string"]},"user_id":{"type":["null","integer"]},"comments":{"type":["null","integer"]},"html_url":{"type":["null","string"]},"assignees":{"type":["null","array"],"items":{"type":["null","integer"]}},"closed_at":{"type":["null","string"],"format":"date-time"},"created_at":{"type":["null","string"],"format":"date-time"},"events_url":{"type":["null","string"]},"labels_url":{"type":["null","string"]},"repository":{"type":["string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"assignee_id":{"type":["null","integer"]},"comments_url":{"type":["null","string"]},"milestone_id":{"type":["null","integer"]},"pull_request":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"diff_url":{"type":["null","string"]},"html_url":{"type":["null","string"]},"patch_url":{"type":["null","string"]}}},"repository_url":{"type":["null","string"]},"active_lock_reason":{"type":["null","string"]},"author_association":{"type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[updated_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@7e8c965b[stream=io.airbyte.protocol.models.AirbyteStream@1c233313[name=github_superset_organizations,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"blog":{"type":["null","string"]},"name":{"type":["null","string"]},"plan":{"type":["null","object"],"properties":{"name":{"type":["null","string"]},"seats":{"type":["null","integer"]},"space":{"type":["null","integer"]},"filled_seats":{"type":["null","integer"]},"private_repos":{"type":["null","integer"]}}},"type":{"type":["null","string"]},"email":{"type":["null","string"]},"login":{"type":["null","string"]},"company":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"location":{"type":["null","string"]},"followers":{"type":["null","integer"]},"following":{"type":["null","integer"]},"hooks_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"disk_usage":{"type":["null","integer"]},"events_url":{"type":["null","string"]},"issues_url":{"type":["null","string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"description":{"type":["null","string"]},"is_verified":{"type":["null","boolean"]},"members_url":{"type":["null","string"]},"public_gists":{"type":["null","integer"]},"public_repos":{"type":["null","integer"]},"billing_email":{"type":["null","string"]},"collaborators":{"type":["null","integer"]},"private_gists":{"type":["null","integer"]},"twitter_username":{"type":["null","string"]},"public_members_url":{"type":["null","string"]},"owned_private_repos":{"type":["null","integer"]},"total_private_repos":{"type":["null","integer"]},"has_repository_projects":{"type":["null","boolean"]},"members_can_create_pages":{"type":["null","boolean"]},"has_organization_projects":{"type":["null","boolean"]},"default_repository_permission":{"type":["null","string"]},"two_factor_requirement_enabled":{"type":["null","boolean"]},"members_can_create_public_pages":{"type":["null","boolean"]},"members_can_create_repositories":{"type":["null","boolean"]},"members_can_create_private_pages":{"type":["null","boolean"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@7bc6e19d[stream=io.airbyte.protocol.models.AirbyteStream@4969077b[name=github_superset_projects,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"body":{"type":["null","string"]},"name":{"type":["null","string"]},"state":{"type":["null","string"]},"number":{"type":["null","integer"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"owner_url":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"creator_id":{"type":["null","integer"]},"repository":{"type":["string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"columns_url":{"type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[updated_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@53cd6e43[stream=io.airbyte.protocol.models.AirbyteStream@56737482[name=github_superset_pull_request_comment_reactions,jsonSchema={"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"content":{"type":["null","string"]},"node_id":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@1a75bdd8[stream=io.airbyte.protocol.models.AirbyteStream@1ba62332[name=github_superset_pull_request_stats,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"merged":{"type":["null","boolean"]},"number":{"type":["null","integer"]},"commits":{"type":["null","integer"]},"node_id":{"type":["null","string"]},"comments":{"type":["null","integer"]},"additions":{"type":["null","integer"]},"deletions":{"type":["null","integer"]},"mergeable":{"type":["null","boolean"]},"merged_by":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"rebaseable":{"type":["null","boolean"]},"repository":{"type":["string"]},"changed_files":{"type":["null","integer"]},"mergeable_state":{"type":["null","string"]},"review_comments":{"type":["null","integer"]},"maintainer_can_modify":{"type":["null","boolean"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@2321e973[stream=io.airbyte.protocol.models.AirbyteStream@6e3b5ef1[name=github_superset_pull_requests,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"base":{"type":["null","object"],"properties":{"ref":{"type":["null","string"]},"sha":{"type":["null","string"]},"label":{"type":["null","string"]},"repo_id":{"type":["null","integer"]},"user_id":{"type":["null","integer"]}}},"body":{"type":["null","string"]},"head":{"type":["null","object"],"properties":{"ref":{"type":["null","string"]},"sha":{"type":["null","string"]},"label":{"type":["null","string"]},"repo_id":{"type":["null","integer"]},"user_id":{"type":["null","integer"]}}},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"draft":{"type":["null","boolean"]},"state":{"type":["null","string"]},"title":{"type":["null","string"]},"_links":{"type":["null","object"],"properties":{"html":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"self":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"issue":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"commits":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"comments":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"statuses":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"review_comment":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"review_comments":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}}}},"labels":{"type":["null","array"],"items":{"type":["null","integer"]}},"locked":{"type":["null","boolean"]},"number":{"type":["null","integer"]},"node_id":{"type":["null","string"]},"diff_url":{"type":["null","string"]},"html_url":{"type":["null","string"]},"assignees":{"type":["null","array"],"items":{"type":["null","integer"]}},"closed_at":{"type":["null","string"],"format":"date-time"},"issue_url":{"type":["null","string"]},"merged_at":{"type":["null","string"],"format":"date-time"},"patch_url":{"type":["null","string"]},"auto_merge":{"type":["null","boolean"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"assignee_id":{"type":["null","integer"]},"commits_url":{"type":["null","string"]},"comments_url":{"type":["null","string"]},"milestone_id":{"type":["null","integer"]},"statuses_url":{"type":["null","string"]},"requested_teams":{"type":["null","array"],"items":{"type":["null","integer"]}},"merge_commit_sha":{"type":["null","string"]},"active_lock_reason":{"type":["null","string"]},"author_association":{"type":["null","string"]},"review_comment_url":{"type":["null","string"]},"requested_reviewers":{"type":["null","array"],"items":{"type":["null","integer"]}},"review_comments_url":{"type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[updated_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@5997a593[stream=io.airbyte.protocol.models.AirbyteStream@7302b220[name=github_superset_releases,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"body":{"type":["null","string"]},"name":{"type":["null","string"]},"draft":{"type":["null","boolean"]},"assets":{"type":["null","array"],"items":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"name":{"type":["null","string"]},"size":{"type":["null","integer"]},"label":{"type":["null","string"]},"state":{"type":["null","string"]},"node_id":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"updated_at":{"type":["null","string"],"format":"date-time"},"uploader_id":{"type":["null","integer"]},"content_type":{"type":["null","string"]},"download_count":{"type":["null","integer"]},"browser_download_url":{"type":["null","string"]}}}},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"tag_name":{"type":["null","string"]},"author_id":{"type":["null","integer"]},"assets_url":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"prerelease":{"type":["null","boolean"]},"repository":{"type":["string"]},"upload_url":{"type":["null","string"]},"tarball_url":{"type":["null","string"]},"zipball_url":{"type":["null","string"]},"published_at":{"type":["null","string"],"format":"date-time"},"target_commitish":{"type":["null","string"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[created_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[created_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@5f44b742[stream=io.airbyte.protocol.models.AirbyteStream@6bbcb68d[name=github_superset_repositories,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"fork":{"type":["null","boolean"]},"name":{"type":["null","string"]},"size":{"type":["null","integer"]},"owner":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"topics":{"type":["null","array"],"items":{"type":["null","string"]}},"git_url":{"type":["null","string"]},"license":{"type":["null","object"],"properties":{"key":{"type":["null","string"]},"url":{"type":["null","string"]},"name":{"type":["null","string"]},"node_id":{"type":["null","string"]},"spdx_id":{"type":["null","string"]},"html_url":{"type":["null","string"]}}},"node_id":{"type":["null","string"]},"private":{"type":["null","boolean"]},"ssh_url":{"type":["null","string"]},"svn_url":{"type":["null","string"]},"archived":{"type":["null","boolean"]},"disabled":{"type":["null","boolean"]},"has_wiki":{"type":["null","boolean"]},"homepage":{"type":["null","string"]},"html_url":{"type":["null","string"]},"keys_url":{"type":["null","string"]},"language":{"type":["null","string"]},"tags_url":{"type":["null","string"]},"blobs_url":{"type":["null","string"]},"clone_url":{"type":["null","string"]},"forks_url":{"type":["null","string"]},"full_name":{"type":["null","string"]},"has_pages":{"type":["null","boolean"]},"hooks_url":{"type":["null","string"]},"pulls_url":{"type":["null","string"]},"pushed_at":{"type":["null","string"],"format":"date-time"},"teams_url":{"type":["null","string"]},"trees_url":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"events_url":{"type":["null","string"]},"has_issues":{"type":["null","boolean"]},"issues_url":{"type":["null","string"]},"labels_url":{"type":["null","string"]},"merges_url":{"type":["null","string"]},"mirror_url":{"type":["null","string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"visibility":{"type":["null","string"]},"archive_url":{"type":["null","string"]},"commits_url":{"type":["null","string"]},"compare_url":{"type":["null","string"]},"description":{"type":["null","string"]},"forks_count":{"type":["null","integer"]},"is_template":{"type":["null","boolean"]},"permissions":{"type":["null","object"],"properties":{"pull":{"type":["null","boolean"]},"push":{"type":["null","boolean"]},"admin":{"type":["null","boolean"]}}},"branches_url":{"type":["null","string"]},"comments_url":{"type":["null","string"]},"contents_url":{"type":["null","string"]},"git_refs_url":{"type":["null","string"]},"git_tags_url":{"type":["null","string"]},"has_projects":{"type":["null","boolean"]},"releases_url":{"type":["null","string"]},"statuses_url":{"type":["null","string"]},"assignees_url":{"type":["null","string"]},"downloads_url":{"type":["null","string"]},"has_downloads":{"type":["null","boolean"]},"languages_url":{"type":["null","string"]},"default_branch":{"type":["null","string"]},"milestones_url":{"type":["null","string"]},"stargazers_url":{"type":["null","string"]},"watchers_count":{"type":["null","integer"]},"deployments_url":{"type":["null","string"]},"git_commits_url":{"type":["null","string"]},"subscribers_url":{"type":["null","string"]},"contributors_url":{"type":["null","string"]},"issue_events_url":{"type":["null","string"]},"stargazers_count":{"type":["null","integer"]},"subscription_url":{"type":["null","string"]},"collaborators_url":{"type":["null","string"]},"issue_comment_url":{"type":["null","string"]},"notifications_url":{"type":["null","string"]},"open_issues_count":{"type":["null","integer"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@f632d54[stream=io.airbyte.protocol.models.AirbyteStream@69161d2a[name=github_superset_review_comments,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"body":{"type":["null","string"]},"line":{"type":["null","integer"]},"path":{"type":["null","string"]},"side":{"type":["null","string"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"_links":{"type":["null","object"],"properties":{"html":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"self":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"pull_request":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}}}},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"position":{"type":["null","integer"]},"commit_id":{"type":["null","string"]},"diff_hunk":{"type":["null","string"]},"created_at":{"type":["null","string"],"format":"date-time"},"repository":{"type":["string"]},"start_line":{"type":["null","integer"]},"start_side":{"type":["null","string"]},"updated_at":{"type":["null","string"],"format":"date-time"},"original_line":{"type":["null","integer"]},"in_reply_to_id":{"type":["null","integer"]},"pull_request_url":{"type":["null","string"]},"original_position":{"type":["null","integer"]},"author_association":{"type":["null","string"]},"original_commit_id":{"type":["null","string"]},"original_start_line":{"type":["null","integer"]},"pull_request_review_id":{"type":["null","integer"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated_at],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[updated_at],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@131249ed[stream=io.airbyte.protocol.models.AirbyteStream@1d148ddc[name=github_superset_reviews,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"body":{"type":["null","string"]},"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"state":{"type":["null","string"]},"_links":{"type":["null","object"],"properties":{"html":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}},"pull_request":{"type":["null","object"],"properties":{"href":{"type":["null","string"]}}}}},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"commit_id":{"type":["null","string"]},"repository":{"type":["string"]},"submitted_at":{"type":["null","string"],"format":"date-time"},"pull_request_url":{"type":["null","string"]},"author_association":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@b8b1aa7[stream=io.airbyte.protocol.models.AirbyteStream@52ada9b1[name=github_superset_stargazers,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"user":{"type":["null","object"],"properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},"user_id":{"type":["null","integer"]},"repository":{"type":["string"]},"starred_at":{"type":["null","string"],"format":"date-time"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[starred_at],sourceDefinedPrimaryKey=[[user_id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[starred_at],destinationSyncMode=append,primaryKey=[[user_id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@53f1cf28[stream=io.airbyte.protocol.models.AirbyteStream@6ab2c2e7[name=github_superset_tags,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"name":{"type":["null","string"]},"commit":{"type":["null","object"],"properties":{"sha":{"type":["null","string"]},"url":{"type":["null","string"]}}},"node_id":{"type":["null","string"]},"repository":{"type":["string"]},"tarball_url":{"type":["null","string"]},"zipball_url":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@39bfa334[stream=io.airbyte.protocol.models.AirbyteStream@6e07593[name=github_superset_users,jsonSchema={"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":["null","integer"]},"url":{"type":["null","string"]},"type":{"type":["null","string"]},"login":{"type":["null","string"]},"node_id":{"type":["null","string"]},"html_url":{"type":["null","string"]},"gists_url":{"type":["null","string"]},"repos_url":{"type":["null","string"]},"avatar_url":{"type":["null","string"]},"events_url":{"type":["null","string"]},"site_admin":{"type":["null","boolean"]},"gravatar_id":{"type":["null","string"]},"starred_url":{"type":["null","string"]},"organization":{"type":["null","string"]},"followers_url":{"type":["null","string"]},"following_url":{"type":["null","string"]},"organizations_url":{"type":["null","string"]},"subscriptions_url":{"type":["null","string"]},"received_events_url":{"type":["null","string"]}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}]],additionalProperties={}]]

Steps to Reproduce

  1. Point the GitHub native connector at a very large repo (like apache/superset), or run many sources at the same time.
  2. Sync job typically fails within a few minutes if the github token has been actively used.

Are you willing to submit a PR?

Probably can't for this one.
@garden-of-delete garden-of-delete added the type/bug Something isn't working label Sep 10, 2021
@sherifnada sherifnada added the area/connectors Connector related issues label Sep 10, 2021
@sherifnada
Copy link
Contributor

sherifnada commented Sep 15, 2021 •
edited

@garden-of-delete I would expect the connector to repeatedly fail but still make incremental progress (unless a single stream is so large that we don't even reach a single state checkpoint). Is that the behavior you're seeing, or does it seem to be replicating the same data over and over?

Is using multiple API tokens an option? The Github connector supports taking multiple PATs as input to load balance quota usage between them

Also, did this just start happening? If so, what was the latest version that the connector was suceeding on?

@annalvova05 annalvova05 self-assigned this Jan 25, 2022
@annalvova05 annalvova05 linked a pull request Feb 2, 2022 that will close this issue
🐛Source Github: remove BAD_GATEWAY code from backoff_time #9999
Merged
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@annalvova05 annalvova05

Labels
area/connectors Connector related issues connectors/source/github connectors/sources-api type/bug Something isn't working
Projects
GL Roadmap
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

🐛Source Github: remove BAD_GATEWAY code from backoff_time airbytehq/airbyte
6 participants
@sherifnada @garden-of-delete @karinakuz @misteryeo @annalvova05 @igrankova