Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
216 lines (213 sloc) 33.5 KB
/home/my/dev/java/jdk1.8.0_51/bin/java -Didea.launcher.port=7532 -Didea.launcher.bin.path=/home/my/dev/idea-IC-145.258.11/bin -Dfile.encoding=UTF-8 -classpath /home/my/dev/java/jdk1.8.0_51/jre/lib/charsets.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/deploy.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/cldrdata.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/dnsns.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/jfxrt.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/localedata.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/nashorn.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/sunec.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/sunjce_provider.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/sunpkcs11.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/ext/zipfs.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/javaws.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/jce.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/jfr.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/jfxswt.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/jsse.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/management-agent.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/plugin.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/resources.jar:/home/my/dev/java/jdk1.8.0_51/jre/lib/rt.jar:/home/my/dev/git/big-lapin/best-lapin-cretin-spark/target/classes:/home/my/.m2/repository/org/apache/spark/spark-core_2.10/1.6.1/spark-core_2.10-1.6.1.jar:/home/my/.m2/repository/org/apache/avro/avro-mapred/1.7.7/avro-mapred-1.7.7-hadoop2.jar:/home/my/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7.jar:/home/my/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/home/my/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7-tests.jar:/home/my/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/home/my/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/home/my/.m2/repository/com/twitter/chill_2.10/0.5.0/chill_2.10-0.5.0.jar:/home/my/.m2/repository/com/esotericsoftware/kryo/kryo/2.21/kryo-2.21.jar:/home/my/.m2/repository/com/esotericsoftware/reflectasm/reflectasm/1.07/reflectasm-1.07-shaded.jar:/home/my/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/home/my/.m2/repository/org/objenesis/objenesis/1.2/objenesis-1.2.jar:/home/my/.m2/repository/com/twitter/chill-java/0.5.0/chill-java-0.5.0.jar:/home/my/.m2/repository/org/apache/xbean/xbean-asm5-shaded/4.4/xbean-asm5-shaded-4.4.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-client/2.2.0/hadoop-client-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-common/2.2.0/hadoop-common-2.2.0.jar:/home/my/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/my/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/home/my/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/home/my/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/home/my/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/home/my/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/home/my/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/home/my/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-auth/2.2.0/hadoop-auth-2.2.0.jar:/home/my/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/home/my/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.2.0/hadoop-hdfs-2.2.0.jar:/home/my/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/2.2.0/hadoop-mapreduce-client-app-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/2.2.0/hadoop-mapreduce-client-common-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-yarn-client/2.2.0/hadoop-yarn-client-2.2.0.jar:/home/my/.m2/repository/com/google/inject/guice/3.0/guice-3.0.jar:/home/my/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/home/my/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/home/my/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-grizzly2/1.9/jersey-test-framework-grizzly2-1.9.jar:/home/my/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-core/1.9/jersey-test-framework-core-1.9.jar:/home/my/.m2/repository/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar:/home/my/.m2/repository/com/sun/jersey/jersey-client/1.9/jersey-client-1.9.jar:/home/my/.m2/repository/com/sun/jersey/jersey-grizzly2/1.9/jersey-grizzly2-1.9.jar:/home/my/.m2/repository/org/glassfish/grizzly/grizzly-http/2.1.2/grizzly-http-2.1.2.jar:/home/my/.m2/repository/org/glassfish/grizzly/grizzly-framework/2.1.2/grizzly-framework-2.1.2.jar:/home/my/.m2/repository/org/glassfish/gmbal/gmbal-api-only/3.0.0-b023/gmbal-api-only-3.0.0-b023.jar:/home/my/.m2/repository/org/glassfish/external/management-api/3.0.0-b012/management-api-3.0.0-b012.jar:/home/my/.m2/repository/org/glassfish/grizzly/grizzly-http-server/2.1.2/grizzly-http-server-2.1.2.jar:/home/my/.m2/repository/org/glassfish/grizzly/grizzly-rcm/2.1.2/grizzly-rcm-2.1.2.jar:/home/my/.m2/repository/org/glassfish/grizzly/grizzly-http-servlet/2.1.2/grizzly-http-servlet-2.1.2.jar:/home/my/.m2/repository/org/glassfish/javax.servlet/3.1/javax.servlet-3.1.jar:/home/my/.m2/repository/com/sun/jersey/jersey-json/1.9/jersey-json-1.9.jar:/home/my/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/home/my/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/home/my/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/home/my/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/home/my/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/home/my/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.8.3/jackson-jaxrs-1.8.3.jar:/home/my/.m2/repository/org/codehaus/jackson/jackson-xc/1.8.3/jackson-xc-1.8.3.jar:/home/my/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.9/jersey-guice-1.9.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/2.2.0/hadoop-yarn-server-common-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.2.0/hadoop-mapreduce-client-shuffle-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.2.0/hadoop-mapreduce-client-core-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.2.0/hadoop-yarn-common-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.2.0/hadoop-mapreduce-client-jobclient-2.2.0.jar:/home/my/.m2/repository/org/apache/hadoop/hadoop-annotations/2.2.0/hadoop-annotations-2.2.0.jar:/home/my/.m2/repository/org/apache/spark/spark-launcher_2.10/1.6.1/spark-launcher_2.10-1.6.1.jar:/home/my/.m2/repository/org/apache/spark/spark-network-common_2.10/1.6.1/spark-network-common_2.10-1.6.1.jar:/home/my/.m2/repository/org/apache/spark/spark-network-shuffle_2.10/1.6.1/spark-network-shuffle_2.10-1.6.1.jar:/home/my/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/home/my/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.4.4/jackson-annotations-2.4.4.jar:/home/my/.m2/repository/org/apache/spark/spark-unsafe_2.10/1.6.1/spark-unsafe_2.10-1.6.1.jar:/home/my/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/home/my/.m2/repository/commons-codec/commons-codec/1.3/commons-codec-1.3.jar:/home/my/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/home/my/.m2/repository/org/apache/curator/curator-recipes/2.4.0/curator-recipes-2.4.0.jar:/home/my/.m2/repository/org/apache/curator/curator-framework/2.4.0/curator-framework-2.4.0.jar:/home/my/.m2/repository/org/apache/curator/curator-client/2.4.0/curator-client-2.4.0.jar:/home/my/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar:/home/my/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/home/my/.m2/repository/com/google/guava/guava/14.0.1/guava-14.0.1.jar:/home/my/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/home/my/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/home/my/.m2/repository/org/apache/commons/commons-math3/3.4.1/commons-math3-3.4.1.jar:/home/my/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/home/my/.m2/repository/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar:/home/my/.m2/repository/org/slf4j/jul-to-slf4j/1.7.10/jul-to-slf4j-1.7.10.jar:/home/my/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.10/jcl-over-slf4j-1.7.10.jar:/home/my/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/my/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar:/home/my/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/home/my/.m2/repository/org/xerial/snappy/snappy-java/1.1.2/snappy-java-1.1.2.jar:/home/my/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/home/my/.m2/repository/org/roaringbitmap/RoaringBitmap/0.5.11/RoaringBitmap-0.5.11.jar:/home/my/.m2/repository/commons-net/commons-net/2.2/commons-net-2.2.jar:/home/my/.m2/repository/com/typesafe/akka/akka-remote_2.10/2.3.11/akka-remote_2.10-2.3.11.jar:/home/my/.m2/repository/com/typesafe/akka/akka-actor_2.10/2.3.11/akka-actor_2.10-2.3.11.jar:/home/my/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/home/my/.m2/repository/io/netty/netty/3.8.0.Final/netty-3.8.0.Final.jar:/home/my/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/home/my/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/home/my/.m2/repository/com/typesafe/akka/akka-slf4j_2.10/2.3.11/akka-slf4j_2.10-2.3.11.jar:/home/my/.m2/repository/org/scala-lang/scala-library/2.10.5/scala-library-2.10.5.jar:/home/my/.m2/repository/org/json4s/json4s-jackson_2.10/3.2.10/json4s-jackson_2.10-3.2.10.jar:/home/my/.m2/repository/org/json4s/json4s-core_2.10/3.2.10/json4s-core_2.10-3.2.10.jar:/home/my/.m2/repository/org/json4s/json4s-ast_2.10/3.2.10/json4s-ast_2.10-3.2.10.jar:/home/my/.m2/repository/org/scala-lang/scalap/2.10.0/scalap-2.10.0.jar:/home/my/.m2/repository/org/scala-lang/scala-compiler/2.10.0/scala-compiler-2.10.0.jar:/home/my/.m2/repository/com/sun/jersey/jersey-server/1.9/jersey-server-1.9.jar:/home/my/.m2/repository/asm/asm/3.1/asm-3.1.jar:/home/my/.m2/repository/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar:/home/my/.m2/repository/org/apache/mesos/mesos/0.21.1/mesos-0.21.1-shaded-protobuf.jar:/home/my/.m2/repository/io/netty/netty-all/4.0.29.Final/netty-all-4.0.29.Final.jar:/home/my/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/home/my/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.2/metrics-core-3.1.2.jar:/home/my/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.2/metrics-jvm-3.1.2.jar:/home/my/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.2/metrics-json-3.1.2.jar:/home/my/.m2/repository/io/dropwizard/metrics/metrics-graphite/3.1.2/metrics-graphite-3.1.2.jar:/home/my/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.4.4/jackson-databind-2.4.4.jar:/home/my/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.4.4/jackson-core-2.4.4.jar:/home/my/.m2/repository/com/fasterxml/jackson/module/jackson-module-scala_2.10/2.4.4/jackson-module-scala_2.10-2.4.4.jar:/home/my/.m2/repository/org/scala-lang/scala-reflect/2.10.4/scala-reflect-2.10.4.jar:/home/my/.m2/repository/com/thoughtworks/paranamer/paranamer/2.6/paranamer-2.6.jar:/home/my/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/home/my/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/home/my/.m2/repository/org/tachyonproject/tachyon-client/0.8.2/tachyon-client-0.8.2.jar:/home/my/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar:/home/my/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/home/my/.m2/repository/org/tachyonproject/tachyon-underfs-hdfs/0.8.2/tachyon-underfs-hdfs-0.8.2.jar:/home/my/.m2/repository/org/tachyonproject/tachyon-underfs-s3/0.8.2/tachyon-underfs-s3-0.8.2.jar:/home/my/.m2/repository/org/tachyonproject/tachyon-underfs-local/0.8.2/tachyon-underfs-local-0.8.2.jar:/home/my/.m2/repository/net/razorvine/pyrolite/4.9/pyrolite-4.9.jar:/home/my/.m2/repository/net/sf/py4j/py4j/0.9/py4j-0.9.jar:/home/my/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:/home/my/dev/idea-IC-145.258.11/lib/idea_rt.jar com.intellij.rt.execution.application.AppMain fr.soat.LapinLePlusCretinWithSparkLocal
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/04/27 21:57:10 INFO SparkContext: Running Spark version 1.6.1
17/04/27 21:57:11 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/04/27 21:57:11 WARN Utils: Your hostname, octopus resolves to a loopback address: 127.0.1.1; using 192.168.1.11 instead (on interface enp7s0)
17/04/27 21:57:11 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
17/04/27 21:57:11 INFO SecurityManager: Changing view acls to: my
17/04/27 21:57:11 INFO SecurityManager: Changing modify acls to: my
17/04/27 21:57:11 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(my); users with modify permissions: Set(my)
17/04/27 21:57:11 INFO Utils: Successfully started service 'sparkDriver' on port 44009.
17/04/27 21:57:12 INFO Slf4jLogger: Slf4jLogger started
17/04/27 21:57:12 INFO Remoting: Starting remoting
17/04/27 21:57:12 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@192.168.1.11:36007]
17/04/27 21:57:12 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 36007.
17/04/27 21:57:12 INFO SparkEnv: Registering MapOutputTracker
17/04/27 21:57:12 INFO SparkEnv: Registering BlockManagerMaster
17/04/27 21:57:12 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-5126c68f-9026-4172-a82e-e89358c9fce1
17/04/27 21:57:12 INFO MemoryStore: MemoryStore started with capacity 1103.3 MB
17/04/27 21:57:12 INFO SparkEnv: Registering OutputCommitCoordinator
17/04/27 21:57:12 INFO Utils: Successfully started service 'SparkUI' on port 4040.
17/04/27 21:57:12 INFO SparkUI: Started SparkUI at http://192.168.1.11:4040
17/04/27 21:57:12 INFO Executor: Starting executor ID driver on host localhost
17/04/27 21:57:12 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 34461.
17/04/27 21:57:12 INFO NettyBlockTransferService: Server created on 34461
17/04/27 21:57:12 INFO BlockManagerMaster: Trying to register BlockManager
17/04/27 21:57:12 INFO BlockManagerMasterEndpoint: Registering block manager localhost:34461 with 1103.3 MB RAM, BlockManagerId(driver, localhost, 34461)
17/04/27 21:57:12 INFO BlockManagerMaster: Registered BlockManager
17/04/27 21:57:13 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 107.7 KB, free 107.7 KB)
17/04/27 21:57:13 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 9.8 KB, free 117.5 KB)
17/04/27 21:57:13 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:34461 (size: 9.8 KB, free: 1103.2 MB)
17/04/27 21:57:13 INFO SparkContext: Created broadcast 0 from textFile at LapinLePlusCretinWithSpark.java:83
17/04/27 21:57:13 INFO FileInputFormat: Total input paths to process : 1
17/04/27 21:57:13 INFO SparkContext: Starting job: reduce at LapinLePlusCretinWithSpark.java:87
17/04/27 21:57:13 INFO DAGScheduler: Got job 0 (reduce at LapinLePlusCretinWithSpark.java:87) with 29 output partitions
17/04/27 21:57:13 INFO DAGScheduler: Final stage: ResultStage 0 (reduce at LapinLePlusCretinWithSpark.java:87)
17/04/27 21:57:13 INFO DAGScheduler: Parents of final stage: List()
17/04/27 21:57:13 INFO DAGScheduler: Missing parents: List()
17/04/27 21:57:13 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[2] at map at LapinLePlusCretinWithSpark.java:86), which has no missing parents
17/04/27 21:57:13 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 4.5 KB, free 122.0 KB)
17/04/27 21:57:13 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.5 KB, free 124.4 KB)
17/04/27 21:57:13 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:34461 (size: 2.5 KB, free: 1103.2 MB)
17/04/27 21:57:13 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1006
17/04/27 21:57:13 INFO DAGScheduler: Submitting 29 missing tasks from ResultStage 0 (MapPartitionsRDD[2] at map at LapinLePlusCretinWithSpark.java:86)
17/04/27 21:57:13 INFO TaskSchedulerImpl: Adding task set 0.0 with 29 tasks
17/04/27 21:57:13 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, partition 0,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:13 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, localhost, partition 1,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:13 INFO TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, localhost, partition 2,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:13 INFO TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, localhost, partition 3,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:13 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
17/04/27 21:57:13 INFO Executor: Running task 1.0 in stage 0.0 (TID 1)
17/04/27 21:57:13 INFO Executor: Running task 2.0 in stage 0.0 (TID 2)
17/04/27 21:57:13 INFO Executor: Running task 3.0 in stage 0.0 (TID 3)
17/04/27 21:57:13 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:33554432+33554432
17/04/27 21:57:13 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:100663296+33554432
17/04/27 21:57:13 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:0+33554432
17/04/27 21:57:13 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:67108864+33554432
17/04/27 21:57:13 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
17/04/27 21:57:13 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
17/04/27 21:57:13 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
17/04/27 21:57:13 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
17/04/27 21:57:13 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
17/04/27 21:57:13 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
17/04/27 21:57:16 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 2407 bytes result sent to driver
17/04/27 21:57:16 INFO Executor: Finished task 2.0 in stage 0.0 (TID 2). 2416 bytes result sent to driver
17/04/27 21:57:16 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 2415 bytes result sent to driver
17/04/27 21:57:16 INFO TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, localhost, partition 4,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:16 INFO Executor: Running task 4.0 in stage 0.0 (TID 4)
17/04/27 21:57:16 INFO TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, localhost, partition 5,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:16 INFO Executor: Running task 5.0 in stage 0.0 (TID 5)
17/04/27 21:57:16 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:167772160+33554432
17/04/27 21:57:16 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:134217728+33554432
17/04/27 21:57:16 INFO TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, localhost, partition 6,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:16 INFO Executor: Finished task 3.0 in stage 0.0 (TID 3). 2416 bytes result sent to driver
17/04/27 21:57:16 INFO Executor: Running task 6.0 in stage 0.0 (TID 6)
17/04/27 21:57:16 INFO TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, localhost, partition 7,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:16 INFO Executor: Running task 7.0 in stage 0.0 (TID 7)
17/04/27 21:57:16 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:201326592+33554432
17/04/27 21:57:16 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:234881024+33554432
17/04/27 21:57:16 INFO TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 3440 ms on localhost (1/29)
17/04/27 21:57:16 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 3465 ms on localhost (2/29)
17/04/27 21:57:16 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 3449 ms on localhost (3/29)
17/04/27 21:57:16 INFO TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 3449 ms on localhost (4/29)
17/04/27 21:57:19 INFO Executor: Finished task 5.0 in stage 0.0 (TID 5). 2415 bytes result sent to driver
17/04/27 21:57:19 INFO TaskSetManager: Starting task 8.0 in stage 0.0 (TID 8, localhost, partition 8,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:19 INFO TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 2644 ms on localhost (5/29)
17/04/27 21:57:19 INFO Executor: Running task 8.0 in stage 0.0 (TID 8)
17/04/27 21:57:19 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:268435456+33554432
17/04/27 21:57:19 INFO Executor: Finished task 4.0 in stage 0.0 (TID 4). 2421 bytes result sent to driver
17/04/27 21:57:19 INFO TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 2658 ms on localhost (6/29)
17/04/27 21:57:19 INFO TaskSetManager: Starting task 9.0 in stage 0.0 (TID 9, localhost, partition 9,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:19 INFO Executor: Running task 9.0 in stage 0.0 (TID 9)
17/04/27 21:57:19 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:301989888+33554432
17/04/27 21:57:19 INFO Executor: Finished task 7.0 in stage 0.0 (TID 7). 2415 bytes result sent to driver
17/04/27 21:57:19 INFO TaskSetManager: Starting task 10.0 in stage 0.0 (TID 10, localhost, partition 10,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:19 INFO TaskSetManager: Finished task 7.0 in stage 0.0 (TID 7) in 2636 ms on localhost (7/29)
17/04/27 21:57:19 INFO Executor: Running task 10.0 in stage 0.0 (TID 10)
17/04/27 21:57:19 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:335544320+33554432
17/04/27 21:57:19 INFO Executor: Finished task 6.0 in stage 0.0 (TID 6). 2415 bytes result sent to driver
17/04/27 21:57:19 INFO TaskSetManager: Starting task 11.0 in stage 0.0 (TID 11, localhost, partition 11,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:19 INFO TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 2677 ms on localhost (8/29)
17/04/27 21:57:19 INFO Executor: Running task 11.0 in stage 0.0 (TID 11)
17/04/27 21:57:19 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:369098752+33554432
17/04/27 21:57:22 INFO Executor: Finished task 8.0 in stage 0.0 (TID 8). 2410 bytes result sent to driver
17/04/27 21:57:22 INFO TaskSetManager: Starting task 12.0 in stage 0.0 (TID 12, localhost, partition 12,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:22 INFO Executor: Running task 12.0 in stage 0.0 (TID 12)
17/04/27 21:57:22 INFO TaskSetManager: Finished task 8.0 in stage 0.0 (TID 8) in 3019 ms on localhost (9/29)
17/04/27 21:57:22 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:402653184+33554432
17/04/27 21:57:22 INFO Executor: Finished task 9.0 in stage 0.0 (TID 9). 2409 bytes result sent to driver
17/04/27 21:57:22 INFO TaskSetManager: Starting task 13.0 in stage 0.0 (TID 13, localhost, partition 13,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:22 INFO Executor: Running task 13.0 in stage 0.0 (TID 13)
17/04/27 21:57:22 INFO TaskSetManager: Finished task 9.0 in stage 0.0 (TID 9) in 3049 ms on localhost (10/29)
17/04/27 21:57:22 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:436207616+33554432
17/04/27 21:57:22 INFO Executor: Finished task 10.0 in stage 0.0 (TID 10). 2415 bytes result sent to driver
17/04/27 21:57:22 INFO TaskSetManager: Starting task 14.0 in stage 0.0 (TID 14, localhost, partition 14,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:22 INFO Executor: Running task 14.0 in stage 0.0 (TID 14)
17/04/27 21:57:22 INFO TaskSetManager: Finished task 10.0 in stage 0.0 (TID 10) in 3071 ms on localhost (11/29)
17/04/27 21:57:22 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:469762048+33554432
17/04/27 21:57:22 INFO Executor: Finished task 11.0 in stage 0.0 (TID 11). 2410 bytes result sent to driver
17/04/27 21:57:22 INFO TaskSetManager: Starting task 15.0 in stage 0.0 (TID 15, localhost, partition 15,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:22 INFO Executor: Running task 15.0 in stage 0.0 (TID 15)
17/04/27 21:57:22 INFO TaskSetManager: Finished task 11.0 in stage 0.0 (TID 11) in 3091 ms on localhost (12/29)
17/04/27 21:57:22 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:503316480+33554432
17/04/27 21:57:25 INFO Executor: Finished task 12.0 in stage 0.0 (TID 12). 2410 bytes result sent to driver
17/04/27 21:57:25 INFO TaskSetManager: Starting task 16.0 in stage 0.0 (TID 16, localhost, partition 16,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:25 INFO TaskSetManager: Finished task 12.0 in stage 0.0 (TID 12) in 2809 ms on localhost (13/29)
17/04/27 21:57:25 INFO Executor: Finished task 14.0 in stage 0.0 (TID 14). 2421 bytes result sent to driver
17/04/27 21:57:25 INFO TaskSetManager: Starting task 17.0 in stage 0.0 (TID 17, localhost, partition 17,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:25 INFO TaskSetManager: Finished task 14.0 in stage 0.0 (TID 14) in 2737 ms on localhost (14/29)
17/04/27 21:57:25 INFO Executor: Running task 16.0 in stage 0.0 (TID 16)
17/04/27 21:57:25 INFO Executor: Finished task 13.0 in stage 0.0 (TID 13). 2415 bytes result sent to driver
17/04/27 21:57:25 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:536870912+33554432
17/04/27 21:57:25 INFO TaskSetManager: Starting task 18.0 in stage 0.0 (TID 18, localhost, partition 18,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:25 INFO TaskSetManager: Finished task 13.0 in stage 0.0 (TID 13) in 2776 ms on localhost (15/29)
17/04/27 21:57:25 INFO Executor: Running task 18.0 in stage 0.0 (TID 18)
17/04/27 21:57:25 INFO Executor: Running task 17.0 in stage 0.0 (TID 17)
17/04/27 21:57:25 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:603979776+33554432
17/04/27 21:57:25 INFO Executor: Finished task 15.0 in stage 0.0 (TID 15). 2416 bytes result sent to driver
17/04/27 21:57:25 INFO TaskSetManager: Starting task 19.0 in stage 0.0 (TID 19, localhost, partition 19,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:25 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:570425344+33554432
17/04/27 21:57:25 INFO TaskSetManager: Finished task 15.0 in stage 0.0 (TID 15) in 2720 ms on localhost (16/29)
17/04/27 21:57:25 INFO Executor: Running task 19.0 in stage 0.0 (TID 19)
17/04/27 21:57:25 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:637534208+33554432
17/04/27 21:57:28 INFO Executor: Finished task 16.0 in stage 0.0 (TID 16). 2409 bytes result sent to driver
17/04/27 21:57:28 INFO TaskSetManager: Starting task 20.0 in stage 0.0 (TID 20, localhost, partition 20,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:28 INFO TaskSetManager: Finished task 16.0 in stage 0.0 (TID 16) in 3057 ms on localhost (17/29)
17/04/27 21:57:28 INFO Executor: Running task 20.0 in stage 0.0 (TID 20)
17/04/27 21:57:28 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:671088640+33554432
17/04/27 21:57:28 INFO Executor: Finished task 17.0 in stage 0.0 (TID 17). 2410 bytes result sent to driver
17/04/27 21:57:28 INFO Executor: Finished task 18.0 in stage 0.0 (TID 18). 2410 bytes result sent to driver
17/04/27 21:57:28 INFO TaskSetManager: Starting task 21.0 in stage 0.0 (TID 21, localhost, partition 21,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:28 INFO Executor: Running task 21.0 in stage 0.0 (TID 21)
17/04/27 21:57:28 INFO TaskSetManager: Starting task 22.0 in stage 0.0 (TID 22, localhost, partition 22,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:28 INFO Executor: Running task 22.0 in stage 0.0 (TID 22)
17/04/27 21:57:28 INFO TaskSetManager: Finished task 17.0 in stage 0.0 (TID 17) in 3075 ms on localhost (18/29)
17/04/27 21:57:28 INFO TaskSetManager: Finished task 18.0 in stage 0.0 (TID 18) in 3068 ms on localhost (19/29)
17/04/27 21:57:28 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:704643072+33554432
17/04/27 21:57:28 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:738197504+33554432
17/04/27 21:57:28 INFO Executor: Finished task 19.0 in stage 0.0 (TID 19). 2415 bytes result sent to driver
17/04/27 21:57:28 INFO TaskSetManager: Starting task 23.0 in stage 0.0 (TID 23, localhost, partition 23,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:28 INFO Executor: Running task 23.0 in stage 0.0 (TID 23)
17/04/27 21:57:28 INFO TaskSetManager: Finished task 19.0 in stage 0.0 (TID 19) in 3063 ms on localhost (20/29)
17/04/27 21:57:28 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:771751936+33554432
17/04/27 21:57:31 INFO Executor: Finished task 21.0 in stage 0.0 (TID 21). 2415 bytes result sent to driver
17/04/27 21:57:31 INFO TaskSetManager: Starting task 24.0 in stage 0.0 (TID 24, localhost, partition 24,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:31 INFO TaskSetManager: Finished task 21.0 in stage 0.0 (TID 21) in 2736 ms on localhost (21/29)
17/04/27 21:57:31 INFO Executor: Running task 24.0 in stage 0.0 (TID 24)
17/04/27 21:57:31 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:805306368+33554432
17/04/27 21:57:31 INFO Executor: Finished task 22.0 in stage 0.0 (TID 22). 2415 bytes result sent to driver
17/04/27 21:57:31 INFO TaskSetManager: Starting task 25.0 in stage 0.0 (TID 25, localhost, partition 25,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:31 INFO TaskSetManager: Finished task 22.0 in stage 0.0 (TID 22) in 2739 ms on localhost (22/29)
17/04/27 21:57:31 INFO Executor: Running task 25.0 in stage 0.0 (TID 25)
17/04/27 21:57:31 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:838860800+33554432
17/04/27 21:57:31 INFO Executor: Finished task 23.0 in stage 0.0 (TID 23). 2404 bytes result sent to driver
17/04/27 21:57:31 INFO TaskSetManager: Starting task 26.0 in stage 0.0 (TID 26, localhost, partition 26,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:31 INFO Executor: Running task 26.0 in stage 0.0 (TID 26)
17/04/27 21:57:31 INFO TaskSetManager: Finished task 23.0 in stage 0.0 (TID 23) in 2737 ms on localhost (23/29)
17/04/27 21:57:31 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:872415232+33554432
17/04/27 21:57:31 INFO Executor: Finished task 20.0 in stage 0.0 (TID 20). 2409 bytes result sent to driver
17/04/27 21:57:31 INFO TaskSetManager: Starting task 27.0 in stage 0.0 (TID 27, localhost, partition 27,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:31 INFO Executor: Running task 27.0 in stage 0.0 (TID 27)
17/04/27 21:57:31 INFO TaskSetManager: Finished task 20.0 in stage 0.0 (TID 20) in 2769 ms on localhost (24/29)
17/04/27 21:57:31 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:905969664+33554432
17/04/27 21:57:34 INFO Executor: Finished task 25.0 in stage 0.0 (TID 25). 2410 bytes result sent to driver
17/04/27 21:57:34 INFO TaskSetManager: Starting task 28.0 in stage 0.0 (TID 28, localhost, partition 28,PROCESS_LOCAL, 2138 bytes)
17/04/27 21:57:34 INFO Executor: Running task 28.0 in stage 0.0 (TID 28)
17/04/27 21:57:34 INFO Executor: Finished task 24.0 in stage 0.0 (TID 24). 2415 bytes result sent to driver
17/04/27 21:57:34 INFO Executor: Finished task 26.0 in stage 0.0 (TID 26). 2409 bytes result sent to driver
17/04/27 21:57:34 INFO HadoopRDD: Input split: file:/home/my/dev/lapins-cretins.txt:939524096+32230156
17/04/27 21:57:34 INFO Executor: Finished task 27.0 in stage 0.0 (TID 27). 2414 bytes result sent to driver
17/04/27 21:57:34 INFO TaskSetManager: Finished task 25.0 in stage 0.0 (TID 25) in 3123 ms on localhost (25/29)
17/04/27 21:57:34 INFO TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 3120 ms on localhost (26/29)
17/04/27 21:57:34 INFO TaskSetManager: Finished task 24.0 in stage 0.0 (TID 24) in 3128 ms on localhost (27/29)
17/04/27 21:57:34 INFO TaskSetManager: Finished task 27.0 in stage 0.0 (TID 27) in 3112 ms on localhost (28/29)
17/04/27 21:57:34 INFO Executor: Finished task 28.0 in stage 0.0 (TID 28). 2416 bytes result sent to driver
17/04/27 21:57:34 INFO TaskSetManager: Finished task 28.0 in stage 0.0 (TID 28) in 623 ms on localhost (29/29)
17/04/27 21:57:34 INFO DAGScheduler: ResultStage 0 (reduce at LapinLePlusCretinWithSpark.java:87) finished in 21,459 s
17/04/27 21:57:34 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/04/27 21:57:34 INFO DAGScheduler: Job 0 finished: reduce at LapinLePlusCretinWithSpark.java:87, took 21,528097 s
Le plus crétin des lapins crétins est :
petit lapin crétin qui fait daaaaaaaaaaa déguisé en pilote de tricycle avec une tronconneuse entre les dents et une balise de chantier dans la main droite
avec un score de : 52
duration : 24835 ms
17/04/27 21:57:34 INFO SparkContext: Invoking stop() from shutdown hook
17/04/27 21:57:35 INFO SparkUI: Stopped Spark web UI at http://192.168.1.11:4040
17/04/27 21:57:35 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/04/27 21:57:35 INFO MemoryStore: MemoryStore cleared
17/04/27 21:57:35 INFO BlockManager: BlockManager stopped
17/04/27 21:57:35 INFO BlockManagerMaster: BlockManagerMaster stopped
17/04/27 21:57:35 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/04/27 21:57:35 INFO SparkContext: Successfully stopped SparkContext
17/04/27 21:57:35 INFO ShutdownHookManager: Shutdown hook called
17/04/27 21:57:35 INFO ShutdownHookManager: Deleting directory /tmp/spark-1c13ec7c-49b7-46fd-987d-9fa5805bea9c
Process finished with exit code 0