Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to create template #20

Closed
anirusharma opened this issue Dec 18, 2018 · 4 comments
Closed

Unable to create template #20

anirusharma opened this issue Dec 18, 2018 · 4 comments
Assignees
@anirusharma

Comments

@anirusharma
Copy link

C:\Users\anirusharma\poc\DataflowTemplates>mvn compile exec:java -Dexec.mainClass=com.google.cloud.teleport.templates.PubsubToPubsub -Dexec.cleanupDaemonThreads=false -Dexec.args="--project=testbatch-211413 --stagingLocation=gs://templates_test_as/staging --tempLocation=gs://templates_test_as/temp --templateLocation=gs://template_data_as/templates/PubsubToPubsub.json --filesToStage=gs://templates_test_as/staging2 --runner=DataflowRunner"
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: windows
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.0
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 0
[INFO] os.detected.classifier: windows-x86_64
[INFO]
[INFO] --------< com.google.cloud.teleport:google-cloud-teleport-java >--------
[INFO] Building Google Cloud Teleport 0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/io/grpc/grpc-core/maven-metadata.xml
Downloaded from central: https://repo.maven.apache.org/maven2/io/grpc/grpc-core/maven-metadata.xml (1.8 kB at 1.4 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/io/netty/netty-codec-http2/maven-metadata.xml
Downloaded from central: https://repo.maven.apache.org/maven2/io/netty/netty-codec-http2/maven-metadata.xml (2.0 kB at 20 kB/s)
Downloading from Apache Snapshots Repository: https://repository.apache.org/content/repositories/snapshots/io/grpc/grpc-core/maven-metadata.xml
Downloading from Sonatype Snapshots Repository: https://oss.sonatype.org/content/repositories/snapshots/io/grpc/grpc-core/maven-metadata.xml
Downloaded from Sonatype Snapshots Repository: https://oss.sonatype.org/content/repositories/snapshots/io/grpc/grpc-core/maven-metadata.xml (802 B at 850 B/s)
Downloading from Sonatype Snapshots Repository: https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-codec-http2/maven-metadata.xml
Downloading from Apache Snapshots Repository: https://repository.apache.org/content/repositories/snapshots/io/netty/netty-codec-http2/maven-metadata.xml
Downloaded from Sonatype Snapshots Repository: https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-codec-http2/maven-metadata.xml (1.5 kB at 6.7 kB/s)
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce) @ google-cloud-teleport-java ---
[INFO] artifact io.grpc:grpc-core: checking for updates from central
[INFO] artifact io.netty:netty-codec-http2: checking for updates from central
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-banned-dependencies) @ google-cloud-teleport-java ---
[INFO]
[INFO] --- protobuf-maven-plugin:0.5.1:compile (default) @ google-cloud-teleport-java ---
[INFO] Compiling 1 proto file(s) to C:\Users\anirusharma\poc\DataflowTemplates\target\generated-sources\protobuf\java
[INFO]
[INFO] --- protobuf-maven-plugin:0.5.1:compile-custom (default) @ google-cloud-teleport-java ---
[INFO] Compiling 1 proto file(s) to C:\Users\anirusharma\poc\DataflowTemplates\target\generated-sources\protobuf\grpc-java
[INFO]
[INFO] --- avro-maven-plugin:1.8.2:schema (default) @ google-cloud-teleport-java ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ google-cloud-teleport-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.6.2:compile (default-compile) @ google-cloud-teleport-java ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 93 source files to C:\Users\anirusharma\poc\DataflowTemplates\target\classes
[INFO] /C:/Users/anirusharma/poc/DataflowTemplates/src/main/java/com/google/cloud/teleport/templates/common/BigQueryConverters.java: Some input files use or override a deprecated API.
[INFO] /C:/Users/anirusharma/poc/DataflowTemplates/src/main/java/com/google/cloud/teleport/templates/common/BigQueryConverters.java: Recompile with -Xlint:deprecation for details.
[INFO] /C:/Users/anirusharma/poc/DataflowTemplates/src/main/java/com/google/cloud/teleport/kafka/connector/KafkaRecordCoder.java: Some input files use unchecked or unsafe operations.
[INFO] /C:/Users/anirusharma/poc/DataflowTemplates/src/main/java/com/google/cloud/teleport/kafka/connector/KafkaRecordCoder.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ google-cloud-teleport-java ---
[WARNING]
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create (Pipeline.java:145)
at com.google.cloud.teleport.templates.PubsubToPubsub.run (PubsubToPubsub.java:149)
at com.google.cloud.teleport.templates.PubsubToPubsub.main (PubsubToPubsub.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create (Pipeline.java:145)
at com.google.cloud.teleport.templates.PubsubToPubsub.run (PubsubToPubsub.java:149)
at com.google.cloud.teleport.templates.PubsubToPubsub.main (PubsubToPubsub.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.RuntimeException: Unable to get application default credentials. Please see https://developers.google.com/accounts/docs/application-default-credentials for details on how to specify credentials. This version of the SDK is dependent on the gcloud core component version 2015.02.05 or newer to be able to get credentials from the currently authorized user via gcloud auth.
at org.apache.beam.sdk.extensions.gcp.auth.NullCredentialInitializer.throwNullCredentialException (NullCredentialInitializer.java:60)
at org.apache.beam.runners.dataflow.util.DataflowTransport.chainHttpRequestInitializer (DataflowTransport.java:99)
at org.apache.beam.runners.dataflow.util.DataflowTransport.newDataflowClient (DataflowTransport.java:76)
at org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions$DataflowClientFactory.create (DataflowPipelineDebugOptions.java:134)
at org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions$DataflowClientFactory.create (DataflowPipelineDebugOptions.java:131)
at org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper (ProxyInvocationHandler.java:592)
at org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault (ProxyInvocationHandler.java:533)
at org.apache.beam.sdk.options.ProxyInvocationHandler.invoke (ProxyInvocationHandler.java:158)
at com.sun.proxy.$Proxy42.getDataflowClient (Unknown Source)
at org.apache.beam.runners.dataflow.DataflowClient.create (DataflowClient.java:41)
at org.apache.beam.runners.dataflow.DataflowRunner. (DataflowRunner.java:338)
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions (DataflowRunner.java:332)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create (Pipeline.java:145)
at com.google.cloud.teleport.templates.PubsubToPubsub.run (PubsubToPubsub.java:149)
at com.google.cloud.teleport.templates.PubsubToPubsub.main (PubsubToPubsub.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:40 min
[INFO] Finished at: 2018-12-18T00:01:47-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions): InvocationTargetException: Unable to get application default credentials. Please see https://developers.google.com/accounts/docs/application-default-credentials for details on how to specify credentials. This version of the SDK is dependent on the gcloud core component version 2015.02.05 or newer to be able to get credentials from the currently authorized user via gcloud auth. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
@anirusharma
Copy link
Author

GOOGLE_APPLICATION_CREDENTIALS was set correctly and gcloud core component version is 2018.12.07 and still getting this issue. please help in resolving

@azurezyq
Copy link
Contributor

azurezyq commented Jun 6, 2019

Are you still experiencing this?
could you please also try running the following command:
gcloud auth application-default login

@gio05
Copy link

gio05 commented Apr 8, 2020

I am having the same issue. I set the GOOGLE_APPLICATION_CREDENTIALS and the gcloud core version is 2020 and also I tried gcloud auth application-default login

@bvolpato
Copy link
Contributor

bvolpato commented Feb 6, 2023

+1, whether need to set GOOGLE_APPLICATION_CREDENTIALS to a service account, run gcloud auth application-default login or run from a GCE instance.

Please check https://cloud.google.com/docs/authentication/application-default-credentials for more details.

This issue has been stale for some time now. Please reopen it if there is a follow up or any related questions.

@bvolpato bvolpato closed this as completed Feb 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@anirusharma anirusharma

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@azurezyq @bvolpato @gio05 @anirusharma