GOOGLE_APPLICATION_CREDENTIALS failed to find key file

[maykiwo]

Hi all,

After have started kafka-connector correctly, I've wanted connect to source GooglePubSub
I'm connect with root account and launch

export GOOGLE_APPLICATION_CREDENTIALS=../../conf/cert-test.json
curl -X POST http://localhost:8086/connectors -H 'content-type: application/json' -d @source-connector-test

Caused by: java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
Do you know what's missing in my conf ?

Thanks

[mdietz94]

Try using an absolute path. Most likely the file does not exist at that location.

…

On Mon, Apr 30, 2018, 11:27 AM maykiwo ***@***.***> wrote: Hi all, After have started kafka-connector correctly, I've wanted connect to source GooglePubSub I'm connect with root account and launch export GOOGLE_APPLICATION_CREDENTIALS=../../conf/cert-test.json curl -X POST http://localhost:8086/connectors -H 'content-type: application/json' -d @source-connector-test Caused by: java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information. Do you know what's missing in my conf ? Thanks — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#158>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvXain978tfMWKWcETLR8ch6pUqnYxNks5tty1-gaJpZM4TsxB4> .

[maykiwo]

I've been trying also absolute path with same error.
The key file has set to 777 for to be sure that can be read for all.

[maykiwo]

Add some informations:

It seems that GOOGLE_APPLICATION_CREDENTIALS must be set before launching Kafka-Connector

Here the correct way :

export GOOGLE_APPLICATION_CREDENTIALS=/usr/hdf/current/kafka-broker/conf/cert-file.json

./connect-distributed.sh ../config/kafka-connect-worker.properties

But with the above method, it seems not possible to launch multiple GooglePubSub, it is rigth?

regards

[kamalaboulhosn]

Apologies for the delay in responding. When you say "launch multiple GlooglePubSub," what do you mean exactly? Launch multiple instances of the connector? Are you trying to launch multiple instances and use different credentials for each? It would be helpful to understand the use case that isn't working for you.

[maykiwo]

Hi @kamalaboulhosn

As I'm understanding kafkaConnector needs one GOOGLE_APPLICATION_CREDENTIAL before we can run it.
So my question is how we can use kafkaConnector if we have multiple GCP projects so multiple credential ?

[lllewellyn]

I have the same question. If I define GOOGLE_APPLICATION_CREDENTIALS to /some/dir/projectA-credentials and now want to publish to a Google projectB, must I have a completely different set of Kafka Connect JVMs so I can supply the projectB credential?

Related question - would it be possible to specify the credentials as a property of the connector config?

[kamalaboulhosn]

The same instance of Kafka connect can only use one credentials file specified via the GOOGLE_APPLICATION_CREDENTIALS environment variable. That means one has to have separate instances of connect running with the environment variable set separately to be able to access Google Cloud Pub/Sub topics or subscriptions that require different credentials. Alternatively, if you are running different instances of the connector within the same connect instance, #187 added support for specifying the credentials file as a config property.