-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tutorial error: Pig version 0.12.0-cdh5.4.2,0.14 is right. #165
Comments
I'm unable to reproduce that error, but it sounds like a jar conflict with the cloudera jars. The tutorial doesn't explicitly call it out, but it's assuming you're using pig-0.13.0, the same as we recommend for the project.clj at the beginning. It looks like you're running a cloudera distribution of pig, but reference the standard version in the project.clj file. If you want to use the cloudera version, I would recommend making both of those the same and changing the hadoop version to a cloudera distribution as well. Another user ran into similar problems in the past and we documented the solution here: https://github.com/Netflix/PigPen/wiki/Hadoop-Versions Let me know if that helps or if there's something else we should look at. |
Throw same error. @.@ (defproject pigpen-demo "0.1.0-SNAPSHOT" |
lein deps :tree [clojure-complete "0.2.3" :scope "test" :exclusions [[org.clojure/clojure]]] [org.apache.pig/pig "0.12.0-cdh5.4.2" :exclusions [io.netty/netty javax.servlet/servlet-api org.apache.zookeeper/zookeeper]] -> [org.apache.hbase/hbase-client "1.0.0-cdh5.4.2" :classifier "tests"] -> [org.apache.hadoop/hadoop-common "2.6.0-cdh5.4.2" :exclusions [javax.servlet.jsp/jsp-api com.sun.jersey/jersey-server javax.servlet/servlet-api tomcat/jasper-compiler tomcat/jasper-runtime]] -> [org.apache.avro/avro "1.7.6-cdh5.4.2"] -> [org.codehaus.jackson/jackson-mapper-asl "1.9.13"] [org.apache.hadoop/hadoop-client "2.6.0-mr1-cdh5.4.2" :exclusions [io.netty/netty]] -> [org.apache.hadoop/hadoop-common "2.6.0-cdh5.4.2" :exclusions [commons-cli commons-httpclient tomcat/jasper-compiler tomcat/jasper-runtime javax.servlet/servlet-api javax.servlet.jsp/jsp-api commons-logging/commons-logging-api jetty/org.mortbay.jetty org.mortbay.jetty/jetty org.mortbay.jetty/jetty-util org.mortbay.jetty/jsp-api-2.1 org.mortbay.jetty/servlet-api-2.5 com.sun.jersey/jersey-core com.sun.jersey/jersey-json com.sun.jersey/jersey-server org.eclipse.jdt/core org.aspectj/aspectjrt org.apache.avro/avro-ipc net.sf.kosmosfs/kfs net.java.dev.jets3t/jets3t]] -> [org.apache.curator/curator-client "2.7.1"] -> [org.slf4j/slf4j-api "1.7.6"] |
/opt/cloudera/parces/jars list accumulo-core-1.6.0.jar |
It's definitely the snappy jar that it's complaining about, but it's weird because cloudera is using snappy-java, not just snappy, so they should be different & shouldn't conflict. From what I can tell, snappy-java doesn't reference When you build the uberjar for your project, does the correct version of |
Follow https://github.com/Netflix/PigPen/wiki/Tutorial
my-script.pig
(defproject pigpen-demo "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.6.0"]
[com.netflix.pigpen/pigpen-pig "0.3.0"]]
:profiles {:dev {:dependencies [[org.apache.pig/pig "0.12.0"]
[org.apache.hadoop/hadoop-core "1.1.2"]]}})
$ pig -i
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Apache Pig version 0.12.0-cdh5.4.2 (rexported)
compiled May 19 2015, 17:03:41
Pig Stack Trace
ERROR 2998: Unhandled internal error. null
java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:635)
at org.apache.pig.impl.PigContext.getClassForAlias(PigContext.java:769)
at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:1491)
at org.apache.pig.parser.LogicalPlanGenerator.func_eval(LogicalPlanGenerator.java:9372)
at org.apache.pig.parser.LogicalPlanGenerator.projectable_expr(LogicalPlanGenerator.java:11051)
at org.apache.pig.parser.LogicalPlanGenerator.var_expr(LogicalPlanGenerator.java:10810)
at org.apache.pig.parser.LogicalPlanGenerator.expr(LogicalPlanGenerator.java:10159)
at org.apache.pig.parser.LogicalPlanGenerator.flatten_generated_item(LogicalPlanGenerator.java:7488)
at org.apache.pig.parser.LogicalPlanGenerator.generate_clause(LogicalPlanGenerator.java:17590)
at org.apache.pig.parser.LogicalPlanGenerator.foreach_plan(LogicalPlanGenerator.java:15982)
at org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:15849)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1933)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1688)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1421)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:354)
at org.apache.pig.PigServer.executeBatch(PigServer.java:379)
at org.apache.pig.PigServer.executeBatch(PigServer.java:365)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:478)
at org.apache.pig.Main.main(Main.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.IllegalArgumentException: No matching method: compress, compiling:(taoensso/nippy/compression.clj:21:22)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6651)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6632)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
at clojure.lang.Compiler$NewInstanceMethod.parse(Compiler.java:8008)
at clojure.lang.Compiler$NewInstanceExpr.build(Compiler.java:7544)
at clojure.lang.Compiler$NewInstanceExpr$DeftypeParser.parse(Compiler.java:7425)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6100)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.analyze(Compiler.java:6406)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5217)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3846)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6642)
at clojure.lang.Compiler.analyze(Compiler.java:6445)
at clojure.lang.Compiler.eval(Compiler.java:6700)
at clojure.lang.Compiler.load(Compiler.java:7130)
at clojure.lang.RT.loadResourceScript(RT.java:370)
at clojure.lang.RT.loadResourceScript(RT.java:361)
at clojure.lang.RT.load(RT.java:440)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5066.invoke(core.clj:5641)
at clojure.core$load.doInvoke(core.clj:5640)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5446)
at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
at clojure.core$load_lib.doInvoke(core.clj:5485)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:626)
at clojure.core$load_libs.doInvoke(core.clj:5528)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:626)
at clojure.core$require.doInvoke(core.clj:5607)
at clojure.lang.RestFn.invoke(RestFn.java:436)
at taoensso.nippy$eval7064$loading__4958__auto____7065.invoke(nippy.clj:1)
at taoensso.nippy$eval7064.invoke(nippy.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6703)
at clojure.lang.Compiler.eval(Compiler.java:6692)
at clojure.lang.Compiler.load(Compiler.java:7130)
at clojure.lang.RT.loadResourceScript(RT.java:370)
at clojure.lang.RT.loadResourceScript(RT.java:361)
at clojure.lang.RT.load(RT.java:440)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5066.invoke(core.clj:5641)
at clojure.core$load.doInvoke(core.clj:5640)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5446)
at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
at clojure.core$load_lib.doInvoke(core.clj:5485)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:626)
at clojure.core$load_libs.doInvoke(core.clj:5524)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:626)
at clojure.core$require.doInvoke(core.clj:5607)
at clojure.lang.RestFn.invoke(RestFn.java:551)
at pigpen.pig.runtime$eval3$loading__4958__auto____4.invoke(runtime.clj:19)
at pigpen.pig.runtime$eval3.invoke(runtime.clj:19)
at clojure.lang.Compiler.eval(Compiler.java:6703)
at clojure.lang.Compiler.eval(Compiler.java:6692)
at clojure.lang.Compiler.load(Compiler.java:7130)
at clojure.lang.RT.loadResourceScript(RT.java:370)
at clojure.lang.RT.loadResourceScript(RT.java:361)
at clojure.lang.RT.load(RT.java:440)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5066.invoke(core.clj:5641)
at clojure.core$load.doInvoke(core.clj:5640)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5446)
at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
at clojure.core$load_lib.doInvoke(core.clj:5485)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:626)
at clojure.core$load_libs.doInvoke(core.clj:5524)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:626)
at clojure.core$require.doInvoke(core.clj:5607)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:379)
at pigpen.PigPenFn.(PigPenFn.java:45)
... 35 more
Caused by: java.lang.IllegalArgumentException: No matching method: compress
at clojure.lang.Compiler$StaticMethodExpr.(Compiler.java:1590)
at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:959)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
... 121 more
The text was updated successfully, but these errors were encountered: