…in mode

…rt-oauth

# Conflicts:
#	airbyte-integrations/connectors/destination-snowflake/src/main/java/io/airbyte/integrations/destination/snowflake/SnowflakeDatabase.java
#	airbyte-integrations/connectors/destination-snowflake/src/test-integration/java/io/airbyte/integrations/destination/snowflake/SnowflakeDestinationIntegrationTest.java

Co-authored-by: timroes <timroes@users.noreply.github.com>

* fix(shopify): wrong type for tax_exemptions

abandoned_checkouts customer tax_exemptions had the wrong field type

* fix(shopify): wrong type for tax_exemptions

abandoned_checkouts customer tax_exemptions had the wrong field type

* bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>

…an 50 (#10588)

* fix execute_in_batch

* add tests

* fix pre-commit config

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Move the feature flag checks to the handler instead of the configuration API. This could have avoid some bug related to the missing flag check in the cloud project.

This is changing the check to see if a connection exist in order to make it more performant and more accurate. It makes sure that the workflow is reachable by trying to query it.

* add timeout to init container command

* add disk usage check into init command

* fix up disk usage checking and logs from init entrypoint

* run format

* test time ranges for cancellations

* try with wait

* fix cancellation on worker restart

* revert for CI testing that the test fails without the retry policy

* revert testing change

* matrix test the different possible cases

* re-enable new retry policy

* switch to no_retry

* switch back to new retry

* paramaterize correctly

* revert to no-retry

* re-enable new retry policy

* speed up test + fixees

* significantly speed up test

* fix ordering

* use multiple task queues in connection manager test

* use versioning for task queue change

* remove sync workflow registration for the connection manager queue

* use more specific example

* respond to parker's comments

* cast timestamp to date

* change test name

* fix corner cases

* fix corner cases 2

* format code

* changed method name

* add return typing

* bump version

* updated spec and def yaml

Co-authored-by: auganbay <auganenu@gmail.com>

as helm templates integers as float64, when using %d, it renders the value of external airbyte.minio.endpoint to "S3_MINIO_ENDPOINT: "http://minio-service:%!d(float64=9000)", therefore needed to be changed to %g

* Add custom survey_ids

* bump version

* Update survey_question schema

* Add changelog

* Allow null objects

* merge master and format

* Make all types safe with NULL and add survey_ids to all streams

* Make additional types safe with NULL

* Make additional types safe with NULL

* One last safe NULL type

* small fixes

* solve conflic

* small fixes

* revert fb wrong commit

* small fb correction

* bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>

…om PR 9027) (#10631)

* Added associations to some CRM Object streams in Hubspot connector

* Added associations in the relevant schemas

* fix eof

* bump connector version

Co-authored-by: ksoenandar <kevin.soenandar@gmail.com>

* added transactions model

* changes

* fix

* few changes

* fix

* added new stream in configured_catalog*.json

* changes

* removed new stream in configured_catalog*.json

* solve small schema issues

* add eof

* bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>

…Audit Activity Events (#10244)

* source(google-workspace-admin-reports): add support for Google Meet Audit activity events

Signed-off-by: Michele Zuccala <michele@zuccala.com>

* remove required fields

* bump connector version

* run format

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>

* stabilize connection manager tests

* just call shutdown once

* another run just so we can see if it's passing

* another run just so we can see if it's passing

* re-disable test

* run another test

* run another test

* run another test

* run another test

…nsient test issue) (#10639)

* log pod state if init pod search times out

* increase test timeout from 5 to 6 minutes to give kube pod process timeout time to trigger

* format

* upgrade temporal from mostly 1.6.0 to 1.8.1

* try bumping GSM to get newer grpc dep

* Revert "try bumping GSM to get newer grpc dep"

This reverts commit d837650.

* upgrade temporal-testing as well

* don't change version for temporal-testing-junit5

* Google Ads #8331 - add network fields to click_view stream schema

* Google Ads #8331 - add segments.ad_network_type to click_view pk according to PR review

* Google Ads #8331 - bump version

* Google Ads #8331 - update definition

Publish metrics for:
- created jobs tagged by release stage
- failed jobs tagged by release stage
- cancelled jobs tagged by release stage
- succeed jobs tagged by release stage

Add,

- attempt_created_by_release_stage
- attempt_failed_by_release_stage
- attempt_succeeded_by_release_stage

…le open portals error (#10235)

* fix cockroachdb connector replication failure due to multiple open portals error

* bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>

* Add try catch to make sure all handlers are closed

* Handle exceptions while initializing writers

* Bumpversion of connectors

* bumpversion in seed

* Fix bigquery denormalized tests

* bumpversion seed of destination bigquery denormalized

* Fix missing key inside map

* Fix onboarding progress links

* Add use-case links to onboarding

* Add new onboarding links

* pipe through to worker

* wip

* pass source and dest def resource reqs to job client

* fix test

* use resource requirements utils to get resource reqs for legacy and new impls

* undo changes to pass sync input to container launcher worker factory

* remove import

* fix hierarchy order of resource requirements

* add nullable annotations

* undo change to test

* format

* use destination resource reqs for normalization and make resource req utils more flexible

* format

* refactor resource requirements utils and add tests

* switch to storing source/dest resource requirements directly on job sync config

* fix tests and javadocs

* use sync input resource requirements for container orchestrator pod

* do not set connection resource reqs to worker reqs

* add overrident requirement utils method + test + comment

Co-authored-by: lmossman <lake@airbyte.io>

Co-authored-by: lmossman <lmossman@users.noreply.github.com>

…PI calls (#10516)

Co-authored-by: sherifnada <sherifnada@users.noreply.github.com>

* Upgrade plop

* Remove scaffolded code

* Build fixes

* Remove scaffolded code

* Revert "Remove scaffolded code"

This reverts commit 3911f52.

* Revert "Remove scaffolded code"

This reverts commit 549f790.

* Remove .gitignore changes

* Remove .gitignore changes

* Update scaffold generated code

* Replace titleCase with capitalCase (#10654)

* Add capitalCase helper

* Replace titleCase with capitalCase

* Update generated scaffold files

Co-authored-by: LiRen Tu <tuliren.git@outlook.com>

…0620)

* close ssh in case of exception

* remove unwanted change

* remove comment

* format

* do not close scanner

* fix semi-colon

* format

…destinations (#10421)

* refactoring to allow testing

* MySQLDestination uses connection property map instead of url arguments

* Update jdbc destinations

* A little more generic

* reset to master

* reset to master

* move to jdbcutils

* Align when multiline

* Align when multiline

* Update postgres to use property map

* Move tests to AbstractJdbcDestinationTest

* clean

* Align when multiline

* return property map

* Add postgres tests

* update clickhouse

* reformat

* reset

* reformat

* fix test

* reformat

* fix bug

* Add mssql tests

* refactor test

* fix oracle destination test

* oracle tests

* fix redshift acceptance test

* Pass string

* Revert "Pass string"

This reverts commit 6978217.

* Double deserialization

* Revert "Double deserialization"

This reverts commit ee8d752.

* try updating json_operations

* Revert "try updating json_operations"

This reverts commit c8022c2.

* json parse

* Revert "json parse"

This reverts commit 11a6725.

* Revert "Revert "Double deserialization""

This reverts commit 213f47a.

* Revert "Revert "Revert "Double deserialization"""

This reverts commit 6682245.

* move to constant

* Add comment

* map can be constant

* Add comment

* move map

* hide in method

* no need to create new map

* no need to create new map

* no need to create new map

* enably mysql test

* Update changelogs

* Update changelog

* update changelog

* Bump versions

* bump version

* disable dbt support

* update spec

* update other oracle tests

* update doc

* bump seed

* fix source test

* update seed spec file

* fix expected spec

Restore the missing update call to temporal.

It was making the update of a schedule to not be effective immediately.

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>

* Fix CockroachDbSource compilation error

* fix test too

* Update Source Zendesk request execution with future requests.

* Revert "Update Source Zendesk request execution with future requests."

This reverts commit 2a3c1f8.

* Add futures stream logics.

* Fix stream

* Fix full refresh streams.

* Update streams.py.
Fix all streams.
Updated schema.

* Add future request unit tests

* Post review fixes.

* Fix broken incremental streams.
Fix SAT.
Remove odd unit tests.

* Comment few unit tests

* Bump docker version

* Ensure AirbyteLogger is thread-safe

- Introduce a global lock to ensure `AirbyteLogger` is thread-safe.
- The `logging` module is thread-safe, however `print` is not, and is currently used. This means that messages sent to stdout can clash if connectors use threading. This is obviously a huge problem when the IPC between the source/destination is stdout!
- A `multiprocessing.Lock` could have been introduced however given that `logging` module is not multiprocess-safe I thought that thread-safety should be first goal.
- IMO the `AirbyteLogger` should be a subclass of the `logging.Logger` so you have thread-safety automatically, however I didn't want to make a huge wholesale change here.

* Revert lock and add deprecation warning instead

Co-authored-by: jrhizor <jrhizor@users.noreply.github.com>

* Add scylla to destination_definitions

* Add woocommerce source

* Update definition id

* Add icon

* update docker repository

* reset to master

* fix version

* generate spec

* Update builds.md

Co-authored-by: girarda <girarda@users.noreply.github.com>

* Fix back link on signup page

* Add and correct uiConfig links

* update postgres source version

* update spec

* #10307 Fixes inconsistent casing on OperationID for Operations API

* update generated doc

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>

* Add connector stage to dropdown value

* Update snowflake destination docs for correct host

* Update snowflake.md

* Update README.md

* Update spec.json

* Update README.md

* Update spec.json

* Update README.md

* Update snowflake.md

* Update spec.json

* Update spec.json

…for adding read only user (#10751)

* add facebook marketing activities stream

* update incremental test

* add overrides for activities specific logic

* formatting

* update readme docs

* remove test limitation

* update dockerfile airbyte version

* correct tests

* bump connector version in config module

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>

Macs with M1 chip can run Airbyte only in dev mode right now, so to make it clear, I added a note about it and moved the hint about M1 chips to the top of the section.

* test: new failures metadata for segment tracking

* new failures metadata for segment tracking

failure_reasons: array of all failures (as json objects) for a job
- for general analytics on failures
main_failure_reason: main failure reason (as json object) for this job
- for operational usage (for Intercom)
- currently this is just the first failure reason chronologically
    - we'll probably to change this when we have more data on how to
determine failure reasons more intelligently

- added an attempt_id to failures so we can group failures by attempt
- removed stacktrace from failures since it's not clear how we'd use
these in an analytics use case (and because segment has a 32kb size
limit for events)

* remove attempt_id

attempt info is already in failure metadata

* explicitly sort failures array chronologically

* replace "unknown" enums with null

note: ImmutableMaps don't allow nulls

* move sorting to the correct place

Increase the temporal retention to 30 days instead of 7. It will help with on call investigation.

* Add count connection functions

* Fix new configRepository queries

- Remove unnecessary joins
- Fix countConnection

* Use existing mock data for tests

…nit container (#10759)

* close ssh tunnel in case of exception

* format

Extract of different events that can happen to a sync into a non temporal related interface.

Co-authored-by: sherifnada <sherifnada@users.noreply.github.com>

Currently, Macs with M1 chips can run Airbyte only in dev mode. I added a note about that.

Add all the simpler queries from https://docs.google.com/document/d/11pEUsHyKUhh4CtV3aReau3SUG-ncEvy6ROJRVln6YB4/edit?usp=sharing.

- Num Pending Jobs
- Num Concurrent Jobs
- Oldest Pending Job
- Oldest Running Job

* Bump Airbyte version from 0.35.43-alpha to 0.35.44-alpha

* Commit.

* Add exception block.

* Why would having try catch work?

* Add logging to figure out.

* Undo all debugging changes.

* Better comments.

Co-authored-by: davinchia <davinchia@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>