Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Class EmrSparkExample - import errors #575

Open
clavvis opened this issue Oct 14, 2017 · 0 comments
Open

Class EmrSparkExample - import errors #575

clavvis opened this issue Oct 14, 2017 · 0 comments
Labels

Comments

@clavvis
Copy link

clavvis commented Oct 14, 2017

I imported dl4j-examples in Eclipse. I use newest code from GitHub. Class EmrSparkExample shows 51 errors. First import errors are on all import com.amazonaws.... Other errors depends on import errors.

pom.xml includes these these aws dependecies:

com.amazonaws
aws-java-sdk-emr
${aws.sdk.version}
provided

	<dependency>
		<groupId>com.amazonaws</groupId>
		<artifactId>aws-java-sdk-s3</artifactId>
		<version>${aws.sdk.version}</version>
		<scope>provided</scope>
	</dependency>

My IDEA doesn't show errors. But after run I get

  1. first run
    Error running 'EmrSparkExample'
    Command line is too long. In order to reduce its length classpath file can be used.
    Would you like to enable classpath file mode for all run configurations of your project?
    I enabled it.
  2. next run:
    java.lang.NoClassDefFoundError: com/amazonaws/auth/AWSCredentials
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
    at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
    at java.lang.Class.getMethod0(Class.java:3018)
    at java.lang.Class.getMethod(Class.java:1784)
    at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
    Caused by: java.lang.ClassNotFoundException: com.amazonaws.auth.AWSCredentials
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 7 more
    Error: A JNI error has occurred, please check your installation and try again
    Exception in thread "main"
    Process finished with exit code 1

In my project I try use only one dependency:

com.amazonaws
aws-java-sdk
1.11.210

I have no one error.

If I run main method I will get exception:
log4j:WARN No appenders could be found for logger (com.amazonaws.AmazonWebServiceClient).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.http.conn.ssl.SSLConnectionSocketFactory.(Ljavax/net/ssl/SSLContext;Ljavax/net/ssl/HostnameVerifier;)V
at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.(SdkTLSSocketFactory.java:56)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:92)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:50)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:38)
at com.amazonaws.http.AmazonHttpClient.(AmazonHttpClient.java:315)
at com.amazonaws.http.AmazonHttpClient.(AmazonHttpClient.java:299)
at com.amazonaws.AmazonWebServiceClient.(AmazonWebServiceClient.java:172)
at com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClient.(AmazonElasticMapReduceClient.java:256)
at com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClientBuilder.build(AmazonElasticMapReduceClientBuilder.java:60)
at com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClientBuilder.build(AmazonElasticMapReduceClientBuilder.java:26)
at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at cz.vhr.examples.dl4j.EmrSparkExample.entryPoint(EmrSparkExample.java:103)
at cz.vhr.examples.dl4j.EmrSparkExample.main(EmrSparkExample.java:83)

Could you help?

Thank you

clavvis

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants