LinkedInAds: account_ids requirement are not necessary and undocumented

[shakedlokits]

Connector Name

source-linkedin-ads

Connector Version

0.2.1

What step the error happened?

Configuring a new connector

Revelant information

When trying to create a new linkedin ads connector, the check container is being ran with:

 docker run --rm --init -i -w /data/32b1da95-991b-4fe6-ab1b-bb5750c50596/0 --log-driver none --name source-linkedin-ads-check-32b1da95-991b-4fe6-ab1b-bb5750c50596-0-sscbm --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-linkedin-ads:0.2.1 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.4 -e WORKER_JOB_ID=32b1da95-991b-4fe6-ab1b-bb5750c50596 airbyte/source-linkedin-ads:0.2.1 check --config source_config.json

The JSON it's being provided is as follows, similar to other connectors:

{"start_date":"2023-01-01","credentials":{"client_id":"<correct client id>","auth_method":"oAuth2.0","client_secret":"<correct client secret>","refresh_token":"<correct refresh token>"}}

It fails with the following log:

2023-07-02 15:54:45 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/32b1da95-991b-4fe6-ab1b-bb5750c50596/0/logs.log
2023-07-02 15:54:45 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.50.4
2023-07-02 15:54:45 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to save workflow id for cancellation
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START CHECK -----
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/source-linkedin-ads:0.2.1 exists...
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/source-linkedin-ads:0.2.1 was found locally.
2023-07-02 15:54:45 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = source-linkedin-ads-check-32b1da95-991b-4fe6-ab1b-bb5750c50596-0-sscbm with resources io.airbyte.config.ResourceRequirements@3b819ca7[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2023-07-02 15:54:45 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/32b1da95-991b-4fe6-ab1b-bb5750c50596/0 --log-driver none --name source-linkedin-ads-check-32b1da95-991b-4fe6-ab1b-bb5750c50596-0-sscbm --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-linkedin-ads:0.2.1 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.4 -e WORKER_JOB_ID=32b1da95-991b-4fe6-ab1b-bb5750c50596 airbyte/source-linkedin-ads:0.2.1 check --config source_config.json
2023-07-02 15:54:45 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-07-02 15:54:45 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - Check failed
2023-07-02 15:54:45 INFO i.a.w.g.DefaultCheckConnectionWorker(run):115 - Check connection job received output: io.airbyte.config.StandardCheckConnectionOutput@6195ab0c[status=failed,message=TypeError("'NoneType' object is not iterable"),additionalProperties={}]
2023-07-02 15:54:45 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END CHECK -----
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - 

And it's because the check of the connector calls the account stream which in turn attempts to pull account_ids from the config

As we are performing oauth with a user, we do not necessarily have this information. Though it could be fetched automatically using the accounts search API..
Or it should be explicitly requested in the documentation.

Relevant log output

2023-07-02 15:54:45 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/32b1da95-991b-4fe6-ab1b-bb5750c50596/0/logs.log
2023-07-02 15:54:45 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.50.4
2023-07-02 15:54:45 INFO i.a.a.c.AirbyteApiClient(retryWithJitterThrows):229 - Attempt 0 to save workflow id for cancellation
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START CHECK -----
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2023-07-02 15:54:45 INFO i.a.c.EnvConfigs(getEnvOrDefault):1228 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/source-linkedin-ads:0.2.1 exists...
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/source-linkedin-ads:0.2.1 was found locally.
2023-07-02 15:54:45 INFO i.a.w.p.DockerProcessFactory(create):139 - Creating docker container = source-linkedin-ads-check-32b1da95-991b-4fe6-ab1b-bb5750c50596-0-sscbm with resources io.airbyte.config.ResourceRequirements@3b819ca7[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2023-07-02 15:54:45 INFO i.a.w.p.DockerProcessFactory(create):192 - Preparing command: docker run --rm --init -i -w /data/32b1da95-991b-4fe6-ab1b-bb5750c50596/0 --log-driver none --name source-linkedin-ads-check-32b1da95-991b-4fe6-ab1b-bb5750c50596-0-sscbm --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-linkedin-ads:0.2.1 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.50.4 -e WORKER_JOB_ID=32b1da95-991b-4fe6-ab1b-bb5750c50596 airbyte/source-linkedin-ads:0.2.1 check --config source_config.json
2023-07-02 15:54:45 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):177 - Reading messages from protocol version 0.2.0
2023-07-02 15:54:45 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):308 - Check failed
2023-07-02 15:54:45 INFO i.a.w.g.DefaultCheckConnectionWorker(run):115 - Check connection job received output: io.airbyte.config.StandardCheckConnectionOutput@6195ab0c[status=failed,message=TypeError("'NoneType' object is not iterable"),additionalProperties={}]
2023-07-02 15:54:45 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - 
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END CHECK -----
2023-07-02 15:54:45 INFO i.a.c.i.LineGobbler(voidCall):149 -