Vertex AI SDK authentication problem

[SetoKaiba]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/google-cloud-java/issues
• Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform

If you are still having issues, please include as much information as possible:

Environment details

1. Specify the API at the beginning of the title. For example, "[vision]: ...").
   General, Core, and Other are also allowed as types
2. OS type and version: windows 10
3. Java version: GraalVM 21.0.1
4. Version(s): vertex 0.1.0

Steps to reproduce

1. Use example here

2. It can't authenticate.
   I set GOOGLE_APPLICATION_CREDENTIALS env var pointing to the service account json.

Code example

https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/send-multimodal-prompts#gemini-send-multimodal-samples-java

Stack trace

Exception in thread "main" com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:116)
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
	at com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:82)
	at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:84)
	at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:148)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
	at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:570)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.RuntimeException: Asynchronous task failed
		at com.google.api.gax.rpc.ServerStreamIterator.hasNext(ServerStreamIterator.java:105)
		at com.google.cloud.vertexai.generativeai.preview.ResponseStreamIteratorWithHistory.hasNext(ResponseStreamIteratorWithHistory.java:37)
		at com.google.cloud.vertexai.generativeai.preview.ResponseHandler.aggregateStreamIntoResponse(ResponseHandler.java:104)
		at com.google.cloud.vertexai.generativeai.preview.GenerativeModel.generateContent(GenerativeModel.java:396)
		at com.google.cloud.vertexai.generativeai.preview.GenerativeModel.generateContent(GenerativeModel.java:457)
		at com.google.cloud.vertexai.generativeai.preview.GenerativeModel.generateContent(GenerativeModel.java:408)
		at org.example.MultimodalMultiImage.multimodalMultiImage(MultimodalMultiImage.java:47)
		at org.example.MultimodalMultiImage.main(MultimodalMultiImage.java:24)
Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
	at io.grpc.Status.asRuntimeException(Status.java:537)
	... 17 more

External references such as API reference guides

• https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/send-multimodal-prompts#gemini-send-multimodal-samples-java

Any additional information below

Following these steps guarantees the quickest resolution possible.

Thanks!

[meltsufin]

Are you able to verify that the credentials pointed to by GOOGLE_APPLICATION_CREDENTIALS are actually valid and have permissions to access the API?

cc: @ZhenyiQ

[SetoKaiba]

I confirmed. The ADC is working. But the GOOGLE_APPLICATION_CREDENTIALS not. I double check with others as well. Others can reproduce this with GOOGLE_APPLICATION_CREDENTIALS way. @meltsufin @ZhenyiQ

[meltsufin]

@ZhenyiQ has been able to reproduce this issue.
@blakeli0 PTAL.

[pandazki]

I also encountered the same issue, and in the end, I was able to temporarily bypass this issue using Transport.REST. Hope this helps @SetoKaiba

[SetoKaiba]

I also encountered the same issue, and in the end, I was able to temporarily bypass this issue using Transport.REST. Hope this helps @SetoKaiba

You can use ADC for authentication. You can still use the sdk.

[ZhenyiQ]

Thanks @SetoKaiba ! This is indeed a bug in the vertexai sdk. We will push a new version to resolve this. Thanks for bringing this to us!

[SetoKaiba]

@blakeli0 @ZhenyiQ @meltsufin Should the defaultScopes be applied to the method public VertexAI(String projectId, String location, GoogleCredentials credentials) as well?