Logged in as: dr.who Application Tools Configuration Local logs Server stacks Server metrics Log Type: stderr Log Upload Time: Wed Oct 07 20:27:13 +0000 2020 Log Length: 195362 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/mnt2/yarn/usercache/hadoop/filecache/3995/__spark_libs__7096009162560369874.zip/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 20/10/07 20:26:20 INFO SignalUtils: Registered signal handler for TERM 20/10/07 20:26:20 INFO SignalUtils: Registered signal handler for HUP 20/10/07 20:26:20 INFO SignalUtils: Registered signal handler for INT 20/10/07 20:26:20 INFO SecurityManager: Changing view acls to: yarn,hadoop 20/10/07 20:26:20 INFO SecurityManager: Changing modify acls to: yarn,hadoop 20/10/07 20:26:20 INFO SecurityManager: Changing view acls groups to: 20/10/07 20:26:20 INFO SecurityManager: Changing modify acls groups to: 20/10/07 20:26:20 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, hadoop); groups with view permissions: Set(); users with modify permissions: Set(yarn, hadoop); groups with modify permissions: Set() 20/10/07 20:26:21 INFO ApplicationMaster: Preparing Local resources 20/10/07 20:26:22 INFO ApplicationMaster: ApplicationAttemptId: appattempt_1601158208025_9843_000002 20/10/07 20:26:22 INFO ApplicationMaster: Starting the user application in a separate Thread 20/10/07 20:26:22 INFO ApplicationMaster: Waiting for spark context initialization... 20/10/07 20:26:22 WARN SchedulerConfGenerator: Job Scheduling Configs will not be in effect as spark.scheduler.mode is not set to FAIR at instantiation time. Continuing without scheduling configs 20/10/07 20:26:22 INFO SparkContext: Running Spark version 2.4.5-amzn-0 20/10/07 20:26:22 INFO SparkContext: Submitted application: delta-streamer-hudi_dms_acc_kafka 20/10/07 20:26:22 INFO SecurityManager: Changing view acls to: yarn,hadoop 20/10/07 20:26:22 INFO SecurityManager: Changing modify acls to: yarn,hadoop 20/10/07 20:26:22 INFO SecurityManager: Changing view acls groups to: 20/10/07 20:26:22 INFO SecurityManager: Changing modify acls groups to: 20/10/07 20:26:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, hadoop); groups with view permissions: Set(); users with modify permissions: Set(yarn, hadoop); groups with modify permissions: Set() 20/10/07 20:26:22 INFO deprecation: mapred.output.compression.codec is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.codec 20/10/07 20:26:22 INFO deprecation: mapred.output.compression.type is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.type 20/10/07 20:26:22 INFO deprecation: mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress 20/10/07 20:26:23 INFO Utils: Successfully started service 'sparkDriver' on port 46843. 20/10/07 20:26:23 INFO SparkEnv: Registering MapOutputTracker 20/10/07 20:26:23 INFO SparkEnv: Registering BlockManagerMaster 20/10/07 20:26:23 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information 20/10/07 20:26:23 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up 20/10/07 20:26:23 INFO DiskBlockManager: Created local directory at /mnt2/yarn/usercache/hadoop/appcache/application_1601158208025_9843/blockmgr-d4bec561-4cbd-4401-92d5-17dae799aa63 20/10/07 20:26:23 INFO DiskBlockManager: Created local directory at /mnt1/yarn/usercache/hadoop/appcache/application_1601158208025_9843/blockmgr-2acb7199-e4c4-48dc-ac6a-4f0cb9d2d9cd 20/10/07 20:26:23 INFO DiskBlockManager: Created local directory at /mnt3/yarn/usercache/hadoop/appcache/application_1601158208025_9843/blockmgr-67860d82-048d-4594-80e8-b1074f5289cf 20/10/07 20:26:23 INFO DiskBlockManager: Created local directory at /mnt/yarn/usercache/hadoop/appcache/application_1601158208025_9843/blockmgr-fb35b0c9-dafe-4334-9176-7d156ab3bd7c 20/10/07 20:26:23 INFO MemoryStore: MemoryStore started with capacity 1608.9 MB 20/10/07 20:26:23 INFO SparkEnv: Registering OutputCommitCoordinator 20/10/07 20:26:23 INFO JettyUtils: Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /jobs, /jobs/json, /jobs/job, /jobs/job/json, /stages, /stages/json, /stages/stage, /stages/stage/json, /stages/pool, /stages/pool/json, /storage, /storage/json, /storage/rdd, /storage/rdd/json, /environment, /environment/json, /executors, /executors/json, /executors/threadDump, /executors/threadDump/json, /static, /, /api, /jobs/job/kill, /stages/stage/kill. 20/10/07 20:26:23 INFO Utils: Successfully started service 'SparkUI' on port 34477. 20/10/07 20:26:23 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://ip-172-31-20-51.ap-southeast-2.compute.internal:34477 20/10/07 20:26:23 INFO YarnClusterScheduler: Created YarnClusterScheduler 20/10/07 20:26:23 INFO SchedulerExtensionServices: Starting Yarn extension services with app application_1601158208025_9843 and attemptId Some(appattempt_1601158208025_9843_000002) 20/10/07 20:26:24 INFO Utils: Using initial executors = 50, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances 20/10/07 20:26:24 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 35667. 20/10/07 20:26:24 INFO NettyBlockTransferService: Server created on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 20/10/07 20:26:24 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 20/10/07 20:26:24 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, ip-172-31-20-51.ap-southeast-2.compute.internal, 35667, None) 20/10/07 20:26:24 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-20-51.ap-southeast-2.compute.internal:35667 with 1608.9 MB RAM, BlockManagerId(driver, ip-172-31-20-51.ap-southeast-2.compute.internal, 35667, None) 20/10/07 20:26:24 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, ip-172-31-20-51.ap-southeast-2.compute.internal, 35667, None) 20/10/07 20:26:24 INFO BlockManager: external shuffle service port = 7337 20/10/07 20:26:24 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, ip-172-31-20-51.ap-southeast-2.compute.internal, 35667, None) 20/10/07 20:26:24 INFO JettyUtils: Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /metrics/json. 20/10/07 20:26:24 INFO EventLoggingListener: Logging events to hdfs:/var/log/spark/apps/application_1601158208025_9843_2 20/10/07 20:26:24 INFO Utils: Using initial executors = 50, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances 20/10/07 20:26:24 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered! 20/10/07 20:26:24 INFO RMProxy: Connecting to ResourceManager at ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal/172.31.23.151:8030 20/10/07 20:26:24 INFO YarnRMClient: Registering the ApplicationMaster 20/10/07 20:26:24 INFO ApplicationMaster: =============================================================================== YARN executor launch context: env: CLASSPATH -> /usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/goodies/lib/emr-spark-goodies.jar:/usr/share/aws/emr/security/conf:/usr/share/aws/emr/security/lib/*:/usr/share/aws/hmclient/lib/aws-glue-datacatalog-spark-client.jar:/usr/share/java/Hive-JSON-Serde/hive-openx-serde.jar:/usr/share/aws/sagemaker-spark-sdk/lib/sagemaker-spark-sdk.jar:/usr/share/aws/emr/s3select/lib/emr-s3-select-spark-connector.jar{{PWD}}{{PWD}}/__spark_conf__{{PWD}}/__spark_libs__/*$HADOOP_CONF_DIR$HADOOP_COMMON_HOME/*$HADOOP_COMMON_HOME/lib/*$HADOOP_HDFS_HOME/*$HADOOP_HDFS_HOME/lib/*$HADOOP_MAPRED_HOME/*$HADOOP_MAPRED_HOME/lib/*$HADOOP_YARN_HOME/*$HADOOP_YARN_HOME/lib/*/usr/lib/hadoop-lzo/lib/*/usr/share/aws/emr/emrfs/conf/usr/share/aws/emr/emrfs/lib/*/usr/share/aws/emr/emrfs/auxlib/*/usr/share/aws/emr/lib/*/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar/usr/share/aws/emr/cloudwatch-sink/lib/*/usr/share/aws/aws-java-sdk/*$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*/usr/lib/hadoop-lzo/lib/*/usr/share/aws/emr/emrfs/conf/usr/share/aws/emr/emrfs/lib/*/usr/share/aws/emr/emrfs/auxlib/*/usr/share/aws/emr/lib/*/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar/usr/share/aws/emr/cloudwatch-sink/lib/*/usr/share/aws/aws-java-sdk/*{{PWD}}/__spark_conf__/__hadoop_conf__ SPARK_YARN_STAGING_DIR -> hdfs://ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal:8020/user/hadoop/.sparkStaging/application_1601158208025_9843 SPARK_USER -> hadoop SPARK_PUBLIC_DNS -> ip-172-31-20-51.ap-southeast-2.compute.internal command: LD_LIBRARY_PATH=\"/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native:$LD_LIBRARY_PATH\" \ {{JAVA_HOME}}/bin/java \ -server \ -Xmx8971m \ '-verbose:gc' \ '-XX:+PrintGCDetails' \ '-XX:+PrintGCDateStamps' \ '-XX:+UseConcMarkSweepGC' \ '-XX:CMSInitiatingOccupancyFraction=70' \ '-XX:MaxHeapFreeRatio=70' \ '-XX:+CMSClassUnloadingEnabled' \ '-XX:OnOutOfMemoryError=kill -9 %p' \ -Djava.io.tmpdir={{PWD}}/tmp \ '-Dspark.driver.port=46843' \ '-Dspark.history.ui.port=18080' \ '-Dspark.ui.port=0' \ -Dspark.yarn.app.container.log.dir= \ org.apache.spark.executor.CoarseGrainedExecutorBackend \ --driver-url \ spark://CoarseGrainedScheduler@ip-172-31-20-51.ap-southeast-2.compute.internal:46843 \ --executor-id \ \ --hostname \ \ --cores \ 4 \ --app-id \ application_1601158208025_9843 \ --user-class-path \ file:$PWD/__app__.jar \ --user-class-path \ file:$PWD/hudi-spark-bundle_2.11-0.6.1-SNAPSHOT.jar \ --user-class-path \ file:$PWD/org.apache.spark_spark-avro_2.11-2.4.4.jar \ --user-class-path \ file:$PWD/org.apache.hadoop_hadoop-aws-2.7.3.jar \ --user-class-path \ file:$PWD/org.spark-project.spark_unused-1.0.0.jar \ --user-class-path \ file:$PWD/org.apache.hadoop_hadoop-common-2.7.3.jar \ --user-class-path \ file:$PWD/com.fasterxml.jackson.core_jackson-databind-2.2.3.jar \ --user-class-path \ file:$PWD/com.fasterxml.jackson.core_jackson-annotations-2.2.3.jar \ --user-class-path \ file:$PWD/com.amazonaws_aws-java-sdk-1.7.4.jar \ --user-class-path \ file:$PWD/org.apache.hadoop_hadoop-annotations-2.7.3.jar \ --user-class-path \ file:$PWD/com.google.guava_guava-11.0.2.jar \ --user-class-path \ file:$PWD/commons-cli_commons-cli-1.2.jar \ --user-class-path \ file:$PWD/org.apache.commons_commons-math3-3.1.1.jar \ --user-class-path \ file:$PWD/xmlenc_xmlenc-0.52.jar \ --user-class-path \ file:$PWD/commons-httpclient_commons-httpclient-3.1.jar \ --user-class-path \ file:$PWD/commons-codec_commons-codec-1.4.jar \ --user-class-path \ file:$PWD/commons-io_commons-io-2.4.jar \ --user-class-path \ file:$PWD/commons-net_commons-net-3.1.jar \ --user-class-path \ file:$PWD/commons-collections_commons-collections-3.2.2.jar \ --user-class-path \ file:$PWD/javax.servlet_servlet-api-2.5.jar \ --user-class-path \ file:$PWD/org.mortbay.jetty_jetty-6.1.26.jar \ --user-class-path \ file:$PWD/org.mortbay.jetty_jetty-util-6.1.26.jar \ --user-class-path \ file:$PWD/com.sun.jersey_jersey-core-1.9.jar \ --user-class-path \ file:$PWD/com.sun.jersey_jersey-json-1.9.jar \ --user-class-path \ file:$PWD/com.sun.jersey_jersey-server-1.9.jar \ --user-class-path \ file:$PWD/commons-logging_commons-logging-1.1.3.jar \ --user-class-path \ file:$PWD/log4j_log4j-1.2.17.jar \ --user-class-path \ file:$PWD/net.java.dev.jets3t_jets3t-0.9.0.jar \ --user-class-path \ file:$PWD/commons-lang_commons-lang-2.6.jar \ --user-class-path \ file:$PWD/commons-configuration_commons-configuration-1.6.jar \ --user-class-path \ file:$PWD/org.slf4j_slf4j-api-1.7.10.jar \ --user-class-path \ file:$PWD/org.codehaus.jackson_jackson-core-asl-1.9.13.jar \ --user-class-path \ file:$PWD/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar \ --user-class-path \ file:$PWD/org.apache.avro_avro-1.7.4.jar \ --user-class-path \ file:$PWD/com.google.protobuf_protobuf-java-2.5.0.jar \ --user-class-path \ file:$PWD/com.google.code.gson_gson-2.2.4.jar \ --user-class-path \ file:$PWD/org.apache.hadoop_hadoop-auth-2.7.3.jar \ --user-class-path \ file:$PWD/com.jcraft_jsch-0.1.42.jar \ --user-class-path \ file:$PWD/org.apache.curator_curator-client-2.7.1.jar \ --user-class-path \ file:$PWD/org.apache.curator_curator-recipes-2.7.1.jar \ --user-class-path \ file:$PWD/com.google.code.findbugs_jsr305-3.0.0.jar \ --user-class-path \ file:$PWD/org.apache.htrace_htrace-core-3.1.0-incubating.jar \ --user-class-path \ file:$PWD/org.apache.zookeeper_zookeeper-3.4.6.jar \ --user-class-path \ file:$PWD/org.apache.commons_commons-compress-1.4.1.jar \ --user-class-path \ file:$PWD/org.codehaus.jettison_jettison-1.1.jar \ --user-class-path \ file:$PWD/com.sun.xml.bind_jaxb-impl-2.2.3-1.jar \ --user-class-path \ file:$PWD/org.codehaus.jackson_jackson-jaxrs-1.9.13.jar \ --user-class-path \ file:$PWD/org.codehaus.jackson_jackson-xc-1.9.13.jar \ --user-class-path \ file:$PWD/javax.xml.bind_jaxb-api-2.2.2.jar \ --user-class-path \ file:$PWD/javax.xml.stream_stax-api-1.0-2.jar \ --user-class-path \ file:$PWD/javax.activation_activation-1.1.jar \ --user-class-path \ file:$PWD/asm_asm-3.2.jar \ --user-class-path \ file:$PWD/org.apache.httpcomponents_httpclient-4.2.5.jar \ --user-class-path \ file:$PWD/org.apache.httpcomponents_httpcore-4.2.5.jar \ --user-class-path \ file:$PWD/com.jamesmurty.utils_java-xmlbuilder-0.4.jar \ --user-class-path \ file:$PWD/commons-digester_commons-digester-1.8.jar \ --user-class-path \ file:$PWD/commons-beanutils_commons-beanutils-core-1.8.0.jar \ --user-class-path \ file:$PWD/commons-beanutils_commons-beanutils-1.7.0.jar \ --user-class-path \ file:$PWD/com.thoughtworks.paranamer_paranamer-2.3.jar \ --user-class-path \ file:$PWD/org.xerial.snappy_snappy-java-1.0.4.1.jar \ --user-class-path \ file:$PWD/org.tukaani_xz-1.0.jar \ --user-class-path \ file:$PWD/org.apache.directory.server_apacheds-kerberos-codec-2.0.0-M15.jar \ --user-class-path \ file:$PWD/org.apache.curator_curator-framework-2.7.1.jar \ --user-class-path \ file:$PWD/org.apache.directory.server_apacheds-i18n-2.0.0-M15.jar \ --user-class-path \ file:$PWD/org.apache.directory.api_api-asn1-api-1.0.0-M20.jar \ --user-class-path \ file:$PWD/org.apache.directory.api_api-util-1.0.0-M20.jar \ --user-class-path \ file:$PWD/org.slf4j_slf4j-log4j12-1.7.10.jar \ --user-class-path \ file:$PWD/io.netty_netty-3.6.2.Final.jar \ --user-class-path \ file:$PWD/javax.servlet.jsp_jsp-api-2.1.jar \ --user-class-path \ file:$PWD/jline_jline-0.9.94.jar \ --user-class-path \ file:$PWD/junit_junit-4.11.jar \ --user-class-path \ file:$PWD/org.hamcrest_hamcrest-core-1.3.jar \ --user-class-path \ file:$PWD/com.fasterxml.jackson.core_jackson-core-2.2.3.jar \ --user-class-path \ file:$PWD/joda-time_joda-time-2.10.6.jar \ 1>/stdout \ 2>/stderr resources: org.apache.hadoop_hadoop-common-2.7.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.hadoop_hadoop-common-2.7.3.jar" } size: 3479293 timestamp: 1602102321274 type: FILE visibility: PRIVATE commons-digester_commons-digester-1.8.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-digester_commons-digester-1.8.jar" } size: 143602 timestamp: 1602102322372 type: FILE visibility: PRIVATE commons-cli_commons-cli-1.2.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-cli_commons-cli-1.2.jar" } size: 41123 timestamp: 1602102321433 type: FILE visibility: PRIVATE commons-httpclient_commons-httpclient-3.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-httpclient_commons-httpclient-3.1.jar" } size: 305001 timestamp: 1602102321507 type: FILE visibility: PRIVATE com.sun.jersey_jersey-server-1.9.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.sun.jersey_jersey-server-1.9.jar" } size: 713089 timestamp: 1602102321731 type: FILE visibility: PRIVATE org.apache.curator_curator-framework-2.7.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.curator_curator-framework-2.7.1.jar" } size: 186273 timestamp: 1602102322913 type: FILE visibility: PRIVATE commons-beanutils_commons-beanutils-1.7.0.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-beanutils_commons-beanutils-1.7.0.jar" } size: 188671 timestamp: 1602102322810 type: FILE visibility: PRIVATE org.codehaus.jettison_jettison-1.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.codehaus.jettison_jettison-1.1.jar" } size: 67758 timestamp: 1602102322151 type: FILE visibility: PRIVATE com.fasterxml.jackson.core_jackson-core-2.2.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.fasterxml.jackson.core_jackson-core-2.2.3.jar" } size: 192699 timestamp: 1602102323112 type: FILE visibility: PRIVATE org.apache.hadoop_hadoop-annotations-2.7.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.hadoop_hadoop-annotations-2.7.3.jar" } size: 40863 timestamp: 1602102321386 type: FILE visibility: PRIVATE com.thoughtworks.paranamer_paranamer-2.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.thoughtworks.paranamer_paranamer-2.3.jar" } size: 29555 timestamp: 1602102322829 type: FILE visibility: PRIVATE net.java.dev.jets3t_jets3t-0.9.0.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/net.java.dev.jets3t_jets3t-0.9.0.jar" } size: 539735 timestamp: 1602102321797 type: FILE visibility: PRIVATE org.codehaus.jackson_jackson-core-asl-1.9.13.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.codehaus.jackson_jackson-core-asl-1.9.13.jar" } size: 232248 timestamp: 1602102321880 type: FILE visibility: PRIVATE org.apache.directory.server_apacheds-kerberos-codec-2.0.0-M15.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.directory.server_apacheds-kerberos-codec-2.0.0-M15.jar" } size: 691479 timestamp: 1602102322893 type: FILE visibility: PRIVATE org.apache.hadoop_hadoop-aws-2.7.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.hadoop_hadoop-aws-2.7.3.jar" } size: 126287 timestamp: 1602102321222 type: FILE visibility: PRIVATE com.sun.xml.bind_jaxb-impl-2.2.3-1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.sun.xml.bind_jaxb-impl-2.2.3-1.jar" } size: 890168 timestamp: 1602102322173 type: FILE visibility: PRIVATE org.apache.hadoop_hadoop-auth-2.7.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.hadoop_hadoop-auth-2.7.3.jar" } size: 94150 timestamp: 1602102321991 type: FILE visibility: PRIVATE joda-time_joda-time-2.10.6.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/joda-time_joda-time-2.10.6.jar" } size: 643778 timestamp: 1602102323132 type: FILE visibility: PRIVATE javax.servlet_servlet-api-2.5.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/javax.servlet_servlet-api-2.5.jar" } size: 105112 timestamp: 1602102321619 type: FILE visibility: PRIVATE __app__.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/hudi-utilities-bundle_2.11-0.6.1-SNAPSHOT.jar" } size: 39867253 timestamp: 1602102320391 type: FILE visibility: PRIVATE org.codehaus.jackson_jackson-xc-1.9.13.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.codehaus.jackson_jackson-xc-1.9.13.jar" } size: 27084 timestamp: 1602102322213 type: FILE visibility: PRIVATE com.google.code.gson_gson-2.2.4.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.google.code.gson_gson-2.2.4.jar" } size: 190432 timestamp: 1602102321970 type: FILE visibility: PRIVATE __spark_conf__ -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/__spark_conf__.zip" } size: 266484 timestamp: 1602102323286 type: ARCHIVE visibility: PRIVATE log4j_log4j-1.2.17.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/log4j_log4j-1.2.17.jar" } size: 489884 timestamp: 1602102321776 type: FILE visibility: PRIVATE commons-collections_commons-collections-3.2.2.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-collections_commons-collections-3.2.2.jar" } size: 588337 timestamp: 1602102321598 type: FILE visibility: PRIVATE junit_junit-4.11.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/junit_junit-4.11.jar" } size: 245039 timestamp: 1602102323072 type: FILE visibility: PRIVATE com.fasterxml.jackson.core_jackson-annotations-2.2.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.fasterxml.jackson.core_jackson-annotations-2.2.3.jar" } size: 33483 timestamp: 1602102321320 type: FILE visibility: PRIVATE org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar" } size: 780664 timestamp: 1602102321903 type: FILE visibility: PRIVATE org.apache.commons_commons-compress-1.4.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.commons_commons-compress-1.4.1.jar" } size: 241367 timestamp: 1602102322132 type: FILE visibility: PRIVATE org.apache.zookeeper_zookeeper-3.4.6.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.zookeeper_zookeeper-3.4.6.jar" } size: 792964 timestamp: 1602102322112 type: FILE visibility: PRIVATE org.apache.commons_commons-math3-3.1.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.commons_commons-math3-3.1.1.jar" } size: 1599627 timestamp: 1602102321459 type: FILE visibility: PRIVATE commons-codec_commons-codec-1.4.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-codec_commons-codec-1.4.jar" } size: 58160 timestamp: 1602102321530 type: FILE visibility: PRIVATE xmlenc_xmlenc-0.52.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/xmlenc_xmlenc-0.52.jar" } size: 15010 timestamp: 1602102321481 type: FILE visibility: PRIVATE commons-io_commons-io-2.4.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-io_commons-io-2.4.jar" } size: 185140 timestamp: 1602102321552 type: FILE visibility: PRIVATE org.apache.spark_spark-avro_2.11-2.4.4.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.spark_spark-avro_2.11-2.4.4.jar" } size: 187318 timestamp: 1602102321199 type: FILE visibility: PRIVATE com.google.code.findbugs_jsr305-3.0.0.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.google.code.findbugs_jsr305-3.0.0.jar" } size: 33031 timestamp: 1602102322069 type: FILE visibility: PRIVATE com.google.protobuf_protobuf-java-2.5.0.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.google.protobuf_protobuf-java-2.5.0.jar" } size: 533455 timestamp: 1602102321948 type: FILE visibility: PRIVATE org.mortbay.jetty_jetty-6.1.26.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.mortbay.jetty_jetty-6.1.26.jar" } size: 539912 timestamp: 1602102321640 type: FILE visibility: PRIVATE com.sun.jersey_jersey-core-1.9.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.sun.jersey_jersey-core-1.9.jar" } size: 458739 timestamp: 1602102321682 type: FILE visibility: PRIVATE org.tukaani_xz-1.0.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.tukaani_xz-1.0.jar" } size: 94672 timestamp: 1602102322871 type: FILE visibility: PRIVATE javax.activation_activation-1.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/javax.activation_activation-1.1.jar" } size: 62983 timestamp: 1602102322271 type: FILE visibility: PRIVATE org.codehaus.jackson_jackson-jaxrs-1.9.13.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.codehaus.jackson_jackson-jaxrs-1.9.13.jar" } size: 18336 timestamp: 1602102322193 type: FILE visibility: PRIVATE org.apache.httpcomponents_httpclient-4.2.5.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.httpcomponents_httpclient-4.2.5.jar" } size: 433368 timestamp: 1602102322312 type: FILE visibility: PRIVATE org.spark-project.spark_unused-1.0.0.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.spark-project.spark_unused-1.0.0.jar" } size: 2777 timestamp: 1602102321245 type: FILE visibility: PRIVATE commons-logging_commons-logging-1.1.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-logging_commons-logging-1.1.3.jar" } size: 62050 timestamp: 1602102321752 type: FILE visibility: PRIVATE commons-beanutils_commons-beanutils-core-1.8.0.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-beanutils_commons-beanutils-core-1.8.0.jar" } size: 206035 timestamp: 1602102322391 type: FILE visibility: PRIVATE com.amazonaws_aws-java-sdk-1.7.4.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.amazonaws_aws-java-sdk-1.7.4.jar" } size: 11948376 timestamp: 1602102321362 type: FILE visibility: PRIVATE org.apache.htrace_htrace-core-3.1.0-incubating.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.htrace_htrace-core-3.1.0-incubating.jar" } size: 1475955 timestamp: 1602102322091 type: FILE visibility: PRIVATE javax.servlet.jsp_jsp-api-2.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/javax.servlet.jsp_jsp-api-2.1.jar" } size: 100636 timestamp: 1602102323032 type: FILE visibility: PRIVATE org.slf4j_slf4j-api-1.7.10.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.slf4j_slf4j-api-1.7.10.jar" } size: 32119 timestamp: 1602102321859 type: FILE visibility: PRIVATE org.apache.avro_avro-1.7.4.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.avro_avro-1.7.4.jar" } size: 303139 timestamp: 1602102321925 type: FILE visibility: PRIVATE commons-configuration_commons-configuration-1.6.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-configuration_commons-configuration-1.6.jar" } size: 298829 timestamp: 1602102321839 type: FILE visibility: PRIVATE org.apache.directory.server_apacheds-i18n-2.0.0-M15.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.directory.server_apacheds-i18n-2.0.0-M15.jar" } size: 44925 timestamp: 1602102322933 type: FILE visibility: PRIVATE org.apache.directory.api_api-asn1-api-1.0.0-M20.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.directory.api_api-asn1-api-1.0.0-M20.jar" } size: 16560 timestamp: 1602102322950 type: FILE visibility: PRIVATE org.apache.curator_curator-recipes-2.7.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.curator_curator-recipes-2.7.1.jar" } size: 270342 timestamp: 1602102322050 type: FILE visibility: PRIVATE javax.xml.stream_stax-api-1.0-2.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/javax.xml.stream_stax-api-1.0-2.jar" } size: 23346 timestamp: 1602102322251 type: FILE visibility: PRIVATE hudi-spark-bundle_2.11-0.6.1-SNAPSHOT.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/hudi-spark-bundle_2.11-0.6.1-SNAPSHOT.jar" } size: 34992024 timestamp: 1602102321175 type: FILE visibility: PRIVATE org.hamcrest_hamcrest-core-1.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.hamcrest_hamcrest-core-1.3.jar" } size: 45024 timestamp: 1602102323091 type: FILE visibility: PRIVATE com.google.guava_guava-11.0.2.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.google.guava_guava-11.0.2.jar" } size: 1648200 timestamp: 1602102321411 type: FILE visibility: PRIVATE javax.xml.bind_jaxb-api-2.2.2.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/javax.xml.bind_jaxb-api-2.2.2.jar" } size: 105134 timestamp: 1602102322233 type: FILE visibility: PRIVATE org.apache.directory.api_api-util-1.0.0-M20.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.directory.api_api-util-1.0.0-M20.jar" } size: 79912 timestamp: 1602102322969 type: FILE visibility: PRIVATE commons-lang_commons-lang-2.6.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-lang_commons-lang-2.6.jar" } size: 284220 timestamp: 1602102321818 type: FILE visibility: PRIVATE org.xerial.snappy_snappy-java-1.0.4.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.xerial.snappy_snappy-java-1.0.4.1.jar" } size: 995968 timestamp: 1602102322851 type: FILE visibility: PRIVATE commons-net_commons-net-3.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/commons-net_commons-net-3.1.jar" } size: 273370 timestamp: 1602102321578 type: FILE visibility: PRIVATE org.slf4j_slf4j-log4j12-1.7.10.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.slf4j_slf4j-log4j12-1.7.10.jar" } size: 8866 timestamp: 1602102322988 type: FILE visibility: PRIVATE com.jamesmurty.utils_java-xmlbuilder-0.4.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.jamesmurty.utils_java-xmlbuilder-0.4.jar" } size: 18490 timestamp: 1602102322352 type: FILE visibility: PRIVATE __spark_libs__ -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/__spark_libs__7096009162560369874.zip" } size: 231412540 timestamp: 1602102319556 type: ARCHIVE visibility: PRIVATE com.jcraft_jsch-0.1.42.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.jcraft_jsch-0.1.42.jar" } size: 185746 timestamp: 1602102322010 type: FILE visibility: PRIVATE io.netty_netty-3.6.2.Final.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/io.netty_netty-3.6.2.Final.jar" } size: 1199572 timestamp: 1602102323010 type: FILE visibility: PRIVATE hive-site.xml -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/hive-site.xml" } size: 2170 timestamp: 1602102323151 type: FILE visibility: PRIVATE com.fasterxml.jackson.core_jackson-databind-2.2.3.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.fasterxml.jackson.core_jackson-databind-2.2.3.jar" } size: 865838 timestamp: 1602102321298 type: FILE visibility: PRIVATE com.sun.jersey_jersey-json-1.9.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/com.sun.jersey_jersey-json-1.9.jar" } size: 147952 timestamp: 1602102321707 type: FILE visibility: PRIVATE asm_asm-3.2.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/asm_asm-3.2.jar" } size: 43398 timestamp: 1602102322291 type: FILE visibility: PRIVATE org.mortbay.jetty_jetty-util-6.1.26.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.mortbay.jetty_jetty-util-6.1.26.jar" } size: 177131 timestamp: 1602102321660 type: FILE visibility: PRIVATE org.apache.httpcomponents_httpcore-4.2.5.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.httpcomponents_httpcore-4.2.5.jar" } size: 227708 timestamp: 1602102322332 type: FILE visibility: PRIVATE jline_jline-0.9.94.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/jline_jline-0.9.94.jar" } size: 87325 timestamp: 1602102323051 type: FILE visibility: PRIVATE org.apache.curator_curator-client-2.7.1.jar -> resource { scheme: "hdfs" host: "ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal" port: 8020 file: "/user/hadoop/.sparkStaging/application_1601158208025_9843/org.apache.curator_curator-client-2.7.1.jar" } size: 69500 timestamp: 1602102322029 type: FILE visibility: PRIVATE =============================================================================== 20/10/07 20:26:24 INFO Utils: Using initial executors = 50, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances 20/10/07 20:26:24 INFO YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(spark://YarnAM@ip-172-31-20-51.ap-southeast-2.compute.internal:46843) 20/10/07 20:26:24 INFO YarnAllocator: Will request 50 executor container(s), each with 4 core(s) and 10653 MB memory (including 1682 MB of overhead) 20/10/07 20:26:24 INFO YarnAllocator: Submitted 50 unlocalized container requests. 20/10/07 20:26:25 INFO ApplicationMaster: Started progress reporter thread with (heartbeat : 3000, initial allocation : 200) intervals 20/10/07 20:26:25 INFO YarnClusterSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0 20/10/07 20:26:25 INFO YarnClusterScheduler: YarnClusterScheduler.postStartHook done 20/10/07 20:26:25 INFO AMRMClientImpl: Received new token for : ip-172-31-30-101.ap-southeast-2.compute.internal:8041 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000002 on host ip-172-31-30-101.ap-southeast-2.compute.internal for executor with ID 1 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000003 on host ip-172-31-30-101.ap-southeast-2.compute.internal for executor with ID 2 20/10/07 20:26:25 INFO YarnAllocator: Received 2 containers from YARN, launching executors on 2 of them. 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO AMRMClientImpl: Received new token for : ip-172-31-17-203.ap-southeast-2.compute.internal:8041 20/10/07 20:26:25 INFO AMRMClientImpl: Received new token for : ip-172-31-19-77.ap-southeast-2.compute.internal:8041 20/10/07 20:26:25 INFO AMRMClientImpl: Received new token for : ip-172-31-20-51.ap-southeast-2.compute.internal:8041 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000004 on host ip-172-31-20-51.ap-southeast-2.compute.internal for executor with ID 3 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000005 on host ip-172-31-20-51.ap-southeast-2.compute.internal for executor with ID 4 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000006 on host ip-172-31-17-203.ap-southeast-2.compute.internal for executor with ID 5 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000007 on host ip-172-31-17-203.ap-southeast-2.compute.internal for executor with ID 6 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000008 on host ip-172-31-19-77.ap-southeast-2.compute.internal for executor with ID 7 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000009 on host ip-172-31-19-77.ap-southeast-2.compute.internal for executor with ID 8 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000010 on host ip-172-31-30-101.ap-southeast-2.compute.internal for executor with ID 9 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000011 on host ip-172-31-30-101.ap-southeast-2.compute.internal for executor with ID 10 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000012 on host ip-172-31-20-51.ap-southeast-2.compute.internal for executor with ID 11 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000013 on host ip-172-31-20-51.ap-southeast-2.compute.internal for executor with ID 12 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000014 on host ip-172-31-17-203.ap-southeast-2.compute.internal for executor with ID 13 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000015 on host ip-172-31-17-203.ap-southeast-2.compute.internal for executor with ID 14 20/10/07 20:26:25 INFO YarnAllocator: Received 12 containers from YARN, launching executors on 12 of them. 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:25 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:26 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000016 on host ip-172-31-19-77.ap-southeast-2.compute.internal for executor with ID 15 20/10/07 20:26:26 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000017 on host ip-172-31-19-77.ap-southeast-2.compute.internal for executor with ID 16 20/10/07 20:26:26 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000018 on host ip-172-31-30-101.ap-southeast-2.compute.internal for executor with ID 17 20/10/07 20:26:26 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000020 on host ip-172-31-20-51.ap-southeast-2.compute.internal for executor with ID 18 20/10/07 20:26:26 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000022 on host ip-172-31-17-203.ap-southeast-2.compute.internal for executor with ID 19 20/10/07 20:26:26 INFO YarnAllocator: Launching container container_1601158208025_9843_02_000023 on host ip-172-31-19-77.ap-southeast-2.compute.internal for executor with ID 20 20/10/07 20:26:26 INFO YarnAllocator: Received 6 containers from YARN, launching executors on 6 of them. 20/10/07 20:26:26 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:26 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:26 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:26 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:26 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:26 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 20/10/07 20:26:29 INFO S3NativeFileSystem: Opening 's3://xxxxx/hudi/conf/hudi-kafka.properties' for reading 20/10/07 20:26:30 WARN SparkContext: Using an existing SparkContext; some configuration may not take effect. 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.19.77:48102) with ID 8 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.19.77:48104) with ID 7 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.30.101:59128) with ID 1 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 8 has registered (new total is 1) 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 7 has registered (new total is 2) 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 1 has registered (new total is 3) 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.30.101:59130) with ID 2 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 2 has registered (new total is 4) 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.17.203:46692) with ID 5 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 5 has registered (new total is 5) 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.30.101:59132) with ID 10 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 10 has registered (new total is 6) 20/10/07 20:26:32 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-19-77.ap-southeast-2.compute.internal:41243 with 5.0 GB RAM, BlockManagerId(8, ip-172-31-19-77.ap-southeast-2.compute.internal, 41243, None) 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.17.203:46694) with ID 13 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 13 has registered (new total is 7) 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.17.203:46696) with ID 6 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 6 has registered (new total is 8) 20/10/07 20:26:32 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-19-77.ap-southeast-2.compute.internal:44393 with 5.0 GB RAM, BlockManagerId(7, ip-172-31-19-77.ap-southeast-2.compute.internal, 44393, None) 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.17.203:46698) with ID 14 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 14 has registered (new total is 9) 20/10/07 20:26:32 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-30-101.ap-southeast-2.compute.internal:32963 with 5.0 GB RAM, BlockManagerId(1, ip-172-31-30-101.ap-southeast-2.compute.internal, 32963, None) 20/10/07 20:26:32 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.30.101:59134) with ID 9 20/10/07 20:26:32 INFO ExecutorAllocationManager: New executor 9 has registered (new total is 10) 20/10/07 20:26:32 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-30-101.ap-southeast-2.compute.internal:42829 with 5.0 GB RAM, BlockManagerId(2, ip-172-31-30-101.ap-southeast-2.compute.internal, 42829, None) 20/10/07 20:26:33 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-17-203.ap-southeast-2.compute.internal:43295 with 5.0 GB RAM, BlockManagerId(5, ip-172-31-17-203.ap-southeast-2.compute.internal, 43295, None) 20/10/07 20:26:33 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-30-101.ap-southeast-2.compute.internal:37747 with 5.0 GB RAM, BlockManagerId(10, ip-172-31-30-101.ap-southeast-2.compute.internal, 37747, None) 20/10/07 20:26:33 INFO ConsumerConfig: ConsumerConfig values: auto.commit.interval.ms = 5000 auto.offset.reset = earliest bootstrap.servers = [http://3.25.149.222:29092] check.crcs = true client.id = connections.max.idle.ms = 540000 default.api.timeout.ms = 60000 enable.auto.commit = true exclude.internal.topics = true fetch.max.bytes = 52428800 fetch.max.wait.ms = 500 fetch.min.bytes = 1 group.id = heartbeat.interval.ms = 3000 interceptor.classes = [] internal.leave.group.on.close = true isolation.level = read_uncommitted key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer max.partition.fetch.bytes = 1048576 max.poll.interval.ms = 300000 max.poll.records = 500 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor] receive.buffer.bytes = 65536 reconnect.backoff.max.ms = 1000 reconnect.backoff.ms = 50 request.timeout.ms = 30000 retry.backoff.ms = 100 sasl.client.callback.handler.class = null sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.login.callback.handler.class = null sasl.login.class = null sasl.login.refresh.buffer.seconds = 300 sasl.login.refresh.min.period.seconds = 60 sasl.login.refresh.window.factor = 0.8 sasl.login.refresh.window.jitter = 0.05 sasl.mechanism = GSSAPI security.protocol = PLAINTEXT send.buffer.bytes = 131072 session.timeout.ms = 10000 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS value.deserializer = class io.confluent.kafka.serializers.KafkaAvroDeserializer 20/10/07 20:26:33 INFO KafkaAvroDeserializerConfig: KafkaAvroDeserializerConfig values: schema.registry.url = [http://3.25.149.222:8081] max.schemas.per.subject = 1000 specific.avro.reader = false 20/10/07 20:26:33 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-17-203.ap-southeast-2.compute.internal:39405 with 5.0 GB RAM, BlockManagerId(13, ip-172-31-17-203.ap-southeast-2.compute.internal, 39405, None) 20/10/07 20:26:33 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.20.51:47904) with ID 4 20/10/07 20:26:33 INFO ExecutorAllocationManager: New executor 4 has registered (new total is 11) 20/10/07 20:26:33 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-17-203.ap-southeast-2.compute.internal:44687 with 5.0 GB RAM, BlockManagerId(6, ip-172-31-17-203.ap-southeast-2.compute.internal, 44687, None) 20/10/07 20:26:33 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-30-101.ap-southeast-2.compute.internal:39303 with 5.0 GB RAM, BlockManagerId(9, ip-172-31-30-101.ap-southeast-2.compute.internal, 39303, None) 20/10/07 20:26:33 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-17-203.ap-southeast-2.compute.internal:45915 with 5.0 GB RAM, BlockManagerId(14, ip-172-31-17-203.ap-southeast-2.compute.internal, 45915, None) 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'validate.non.null' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.datasource.write.partitionpath.field' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.compact.inline' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.delete.shuffle.parallelism' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.datasource.write.recordkey.field' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.upsert.shuffle.parallelism' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.datasource.write.keygenerator.class' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.deltastreamer.source.kafka.topic' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.deltastreamer.schemaprovider.registry.url' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.insert.shuffle.parallelism' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.datasource.write.precombine.field' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.embed.timeline.server' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.bulkinsert.shuffle.parallelism' was supplied but isn't a known config. 20/10/07 20:26:33 WARN ConsumerConfig: The configuration 'hoodie.filesystem.view.type' was supplied but isn't a known config. 20/10/07 20:26:33 INFO AppInfoParser: Kafka version : 2.0.0 20/10/07 20:26:33 INFO AppInfoParser: Kafka commitId : 3402a8361b734732 20/10/07 20:26:33 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.20.51:47906) with ID 3 20/10/07 20:26:33 INFO ExecutorAllocationManager: New executor 3 has registered (new total is 12) 20/10/07 20:26:34 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.20.51:47908) with ID 11 20/10/07 20:26:34 INFO ExecutorAllocationManager: New executor 11 has registered (new total is 13) 20/10/07 20:26:34 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-20-51.ap-southeast-2.compute.internal:39129 with 5.0 GB RAM, BlockManagerId(4, ip-172-31-20-51.ap-southeast-2.compute.internal, 39129, None) 20/10/07 20:26:34 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.20.51:47910) with ID 12 20/10/07 20:26:34 INFO ExecutorAllocationManager: New executor 12 has registered (new total is 14) 20/10/07 20:26:34 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-20-51.ap-southeast-2.compute.internal:45015 with 5.0 GB RAM, BlockManagerId(3, ip-172-31-20-51.ap-southeast-2.compute.internal, 45015, None) 20/10/07 20:26:34 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.19.77:48118) with ID 15 20/10/07 20:26:34 INFO ExecutorAllocationManager: New executor 15 has registered (new total is 15) 20/10/07 20:26:34 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.19.77:48116) with ID 16 20/10/07 20:26:34 INFO ExecutorAllocationManager: New executor 16 has registered (new total is 16) 20/10/07 20:26:34 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.17.203:46704) with ID 19 20/10/07 20:26:34 INFO ExecutorAllocationManager: New executor 19 has registered (new total is 17) 20/10/07 20:26:34 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.19.77:48120) with ID 20 20/10/07 20:26:34 INFO ExecutorAllocationManager: New executor 20 has registered (new total is 18) 20/10/07 20:26:34 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-20-51.ap-southeast-2.compute.internal:34719 with 5.0 GB RAM, BlockManagerId(11, ip-172-31-20-51.ap-southeast-2.compute.internal, 34719, None) 20/10/07 20:26:35 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-20-51.ap-southeast-2.compute.internal:44249 with 5.0 GB RAM, BlockManagerId(12, ip-172-31-20-51.ap-southeast-2.compute.internal, 44249, None) 20/10/07 20:26:35 INFO Metadata: Cluster ID: cA3sXVaIR-qlM1MPNNYnCw 20/10/07 20:26:35 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-19-77.ap-southeast-2.compute.internal:45515 with 5.0 GB RAM, BlockManagerId(15, ip-172-31-19-77.ap-southeast-2.compute.internal, 45515, None) 20/10/07 20:26:35 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-17-203.ap-southeast-2.compute.internal:46075 with 5.0 GB RAM, BlockManagerId(19, ip-172-31-17-203.ap-southeast-2.compute.internal, 46075, None) 20/10/07 20:26:35 WARN KafkaUtils: overriding enable.auto.commit to false for executor 20/10/07 20:26:35 WARN KafkaUtils: overriding auto.offset.reset to none for executor 20/10/07 20:26:35 ERROR KafkaUtils: group.id is null, you should probably set it 20/10/07 20:26:35 WARN KafkaUtils: overriding executor group.id to spark-executor-null 20/10/07 20:26:35 WARN KafkaUtils: overriding receive.buffer.bytes to 65536 see KAFKA-3135 20/10/07 20:26:35 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-19-77.ap-southeast-2.compute.internal:39787 with 5.0 GB RAM, BlockManagerId(16, ip-172-31-19-77.ap-southeast-2.compute.internal, 39787, None) 20/10/07 20:26:35 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-19-77.ap-southeast-2.compute.internal:46051 with 5.0 GB RAM, BlockManagerId(20, ip-172-31-19-77.ap-southeast-2.compute.internal, 46051, None) 20/10/07 20:26:35 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.30.101:59146) with ID 17 20/10/07 20:26:35 INFO ExecutorAllocationManager: New executor 17 has registered (new total is 19) 20/10/07 20:26:36 INFO SparkContext: Starting job: isEmpty at AvroConversionUtils.scala:59 20/10/07 20:26:36 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-30-101.ap-southeast-2.compute.internal:39987 with 5.0 GB RAM, BlockManagerId(17, ip-172-31-30-101.ap-southeast-2.compute.internal, 39987, None) 20/10/07 20:26:36 INFO DAGScheduler: Got job 0 (isEmpty at AvroConversionUtils.scala:59) with 1 output partitions 20/10/07 20:26:36 INFO DAGScheduler: Final stage: ResultStage 0 (isEmpty at AvroConversionUtils.scala:59) 20/10/07 20:26:36 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:26:36 INFO DAGScheduler: Missing parents: List() 20/10/07 20:26:36 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[1] at map at AvroKafkaSource.java:74), which has no missing parents 20/10/07 20:26:36 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.31.20.51:47918) with ID 18 20/10/07 20:26:36 INFO ExecutorAllocationManager: New executor 18 has registered (new total is 20) 20/10/07 20:26:36 INFO YarnAllocator: Driver requested a total number of 1 executor(s). 20/10/07 20:26:36 INFO YarnAllocator: Canceling requests for 30 executor container(s) to have a new desired total 1 executors. 20/10/07 20:26:37 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 5.2 KB, free 1608.9 MB) 20/10/07 20:26:37 INFO BlockManagerMasterEndpoint: Registering block manager ip-172-31-20-51.ap-southeast-2.compute.internal:37377 with 5.0 GB RAM, BlockManagerId(18, ip-172-31-20-51.ap-southeast-2.compute.internal, 37377, None) 20/10/07 20:26:38 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 3.2 KB, free 1608.9 MB) 20/10/07 20:26:38 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 3.2 KB, free: 1608.9 MB) 20/10/07 20:26:38 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:38 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[1] at map at AvroKafkaSource.java:74) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:26:38 INFO YarnClusterScheduler: Adding task set 0.0 with 1 tasks 20/10/07 20:26:38 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 4, partition 0, PROCESS_LOCAL, 7774 bytes) 20/10/07 20:26:39 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 (size: 3.2 KB, free: 5.0 GB) 20/10/07 20:26:41 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 3532 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 4) (1/1) 20/10/07 20:26:41 INFO YarnClusterScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool 20/10/07 20:26:41 INFO DAGScheduler: ResultStage 0 (isEmpty at AvroConversionUtils.scala:59) finished in 5.145 s 20/10/07 20:26:41 INFO DAGScheduler: Job 0 finished: isEmpty at AvroConversionUtils.scala:59, took 5.526745 s 20/10/07 20:26:41 INFO YarnAllocator: Driver requested a total number of 0 executor(s). 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 14 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 18 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 13 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 8 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 22 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 5 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 16 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 1 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 15 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 20 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 9 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 7 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 23 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 2 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 19 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 6 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 11 20/10/07 20:26:42 INFO BlockManagerInfo: Removed broadcast_0_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 3.2 KB, free: 1608.9 MB) 20/10/07 20:26:42 INFO BlockManagerInfo: Removed broadcast_0_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 in memory (size: 3.2 KB, free: 5.0 GB) 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 12 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 10 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 21 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 0 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 3 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 17 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 4 20/10/07 20:26:42 INFO ContextCleaner: Cleaned accumulator 24 20/10/07 20:26:43 INFO SharedState: loading hive config file: file:/mnt1/yarn/usercache/hadoop/appcache/application_1601158208025_9843/container_1601158208025_9843_02_000001/hive-site.xml 20/10/07 20:26:43 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('hdfs:///user/spark/warehouse'). 20/10/07 20:26:43 INFO SharedState: Warehouse path is 'hdfs:///user/spark/warehouse'. 20/10/07 20:26:43 INFO JettyUtils: Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /SQL. 20/10/07 20:26:43 INFO JettyUtils: Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /SQL/json. 20/10/07 20:26:43 INFO JettyUtils: Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /SQL/execution. 20/10/07 20:26:43 INFO JettyUtils: Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /SQL/execution/json. 20/10/07 20:26:43 INFO JettyUtils: Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /static/sql. 20/10/07 20:26:44 INFO StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint 20/10/07 20:26:46 INFO CodeGenerator: Code generated in 283.715491 ms 20/10/07 20:26:46 INFO CodeGenerator: Code generated in 73.595611 ms 20/10/07 20:26:46 INFO CodeGenerator: Code generated in 48.692316 ms 20/10/07 20:26:46 INFO SparkContext: Starting job: showString at DebeziumTransformer.java:46 20/10/07 20:26:46 INFO DAGScheduler: Got job 1 (showString at DebeziumTransformer.java:46) with 1 output partitions 20/10/07 20:26:46 INFO DAGScheduler: Final stage: ResultStage 1 (showString at DebeziumTransformer.java:46) 20/10/07 20:26:46 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:26:46 INFO DAGScheduler: Missing parents: List() 20/10/07 20:26:46 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[10] at showString at DebeziumTransformer.java:46), which has no missing parents 20/10/07 20:26:46 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 46.6 KB, free 1608.9 MB) 20/10/07 20:26:46 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 15.4 KB, free 1608.8 MB) 20/10/07 20:26:46 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 15.4 KB, free: 1608.9 MB) 20/10/07 20:26:46 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:46 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[10] at showString at DebeziumTransformer.java:46) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:26:46 INFO YarnClusterScheduler: Adding task set 1.0 with 1 tasks 20/10/07 20:26:46 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 0, PROCESS_LOCAL, 7883 bytes) 20/10/07 20:26:46 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 15.4 KB, free: 5.0 GB) 20/10/07 20:26:50 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 3884 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/1) 20/10/07 20:26:50 INFO YarnClusterScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool 20/10/07 20:26:50 INFO DAGScheduler: ResultStage 1 (showString at DebeziumTransformer.java:46) finished in 3.921 s 20/10/07 20:26:50 INFO DAGScheduler: Job 1 finished: showString at DebeziumTransformer.java:46, took 3.933096 s 20/10/07 20:26:50 INFO CodeGenerator: Code generated in 24.132845 ms 20/10/07 20:26:50 INFO CodeGenerator: Code generated in 20.882773 ms 20/10/07 20:26:50 INFO SparkContext: Starting job: isEmpty at DeltaSync.java:349 20/10/07 20:26:50 INFO DAGScheduler: Got job 2 (isEmpty at DeltaSync.java:349) with 1 output partitions 20/10/07 20:26:50 INFO DAGScheduler: Final stage: ResultStage 2 (isEmpty at DeltaSync.java:349) 20/10/07 20:26:50 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:26:50 INFO DAGScheduler: Missing parents: List() 20/10/07 20:26:50 INFO DAGScheduler: Submitting ResultStage 2 (MapPartitionsRDD[18] at mapPartitions at AvroConversionUtils.scala:45), which has no missing parents 20/10/07 20:26:50 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 60.9 KB, free 1608.8 MB) 20/10/07 20:26:50 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 21.2 KB, free 1608.8 MB) 20/10/07 20:26:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 21.2 KB, free: 1608.9 MB) 20/10/07 20:26:50 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:50 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (MapPartitionsRDD[18] at mapPartitions at AvroConversionUtils.scala:45) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:26:50 INFO YarnClusterScheduler: Adding task set 2.0 with 1 tasks 20/10/07 20:26:50 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 2, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 0, PROCESS_LOCAL, 7883 bytes) 20/10/07 20:26:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 21.2 KB, free: 5.0 GB) 20/10/07 20:26:50 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 2) in 229 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/1) 20/10/07 20:26:50 INFO YarnClusterScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool 20/10/07 20:26:50 INFO DAGScheduler: ResultStage 2 (isEmpty at DeltaSync.java:349) finished in 0.242 s 20/10/07 20:26:50 INFO DAGScheduler: Job 2 finished: isEmpty at DeltaSync.java:349, took 0.247132 s 20/10/07 20:26:51 INFO Javalin: __ __ _ / /____ _ _ __ ____ _ / /(_)____ __ / // __ `/| | / // __ `// // // __ \ / /_/ // /_/ / | |/ // /_/ // // // / / / \____/ \__,_/ |___/ \__,_//_//_//_/ /_/ https://javalin.io/documentation 20/10/07 20:26:51 INFO Javalin: Starting Javalin ... 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 44 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 84 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 69 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 67 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 29 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 62 20/10/07 20:26:51 INFO BlockManagerInfo: Removed broadcast_2_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 21.2 KB, free: 1608.9 MB) 20/10/07 20:26:51 INFO BlockManagerInfo: Removed broadcast_2_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 21.2 KB, free: 5.0 GB) 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 78 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 79 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 38 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 34 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 37 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 33 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 27 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 47 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 82 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 76 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 41 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 30 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 83 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 81 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 51 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 48 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 42 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 68 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 32 20/10/07 20:26:51 INFO BlockManagerInfo: Removed broadcast_1_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 15.4 KB, free: 5.0 GB) 20/10/07 20:26:51 INFO BlockManagerInfo: Removed broadcast_1_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 15.4 KB, free: 1608.9 MB) 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 54 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 31 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 64 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 45 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 73 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 28 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 63 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 36 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 85 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 43 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 50 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 46 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 49 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 35 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 65 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 53 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 40 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 80 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 71 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 26 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 39 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 52 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 66 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 25 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 75 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 72 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 74 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 55 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 77 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 86 20/10/07 20:26:51 INFO ContextCleaner: Cleaned accumulator 70 20/10/07 20:26:51 INFO Javalin: Listening on http://localhost:35045/ 20/10/07 20:26:51 INFO Javalin: Javalin started in 257ms \o/ 20/10/07 20:26:51 INFO SparkContext: Starting job: isEmpty at DeltaSync.java:384 20/10/07 20:26:51 INFO DAGScheduler: Got job 3 (isEmpty at DeltaSync.java:384) with 1 output partitions 20/10/07 20:26:51 INFO DAGScheduler: Final stage: ResultStage 3 (isEmpty at DeltaSync.java:384) 20/10/07 20:26:51 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:26:51 INFO DAGScheduler: Missing parents: List() 20/10/07 20:26:51 INFO DAGScheduler: Submitting ResultStage 3 (MapPartitionsRDD[19] at map at DeltaSync.java:356), which has no missing parents 20/10/07 20:26:51 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 64.1 KB, free 1608.8 MB) 20/10/07 20:26:51 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 23.1 KB, free 1608.8 MB) 20/10/07 20:26:51 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 23.1 KB, free: 1608.9 MB) 20/10/07 20:26:51 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:51 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (MapPartitionsRDD[19] at map at DeltaSync.java:356) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:26:51 INFO YarnClusterScheduler: Adding task set 3.0 with 1 tasks 20/10/07 20:26:51 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 3, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 0, PROCESS_LOCAL, 7883 bytes) 20/10/07 20:26:51 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 23.1 KB, free: 5.0 GB) 20/10/07 20:26:51 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 3) in 187 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/1) 20/10/07 20:26:51 INFO YarnClusterScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool 20/10/07 20:26:51 INFO DAGScheduler: ResultStage 3 (isEmpty at DeltaSync.java:384) finished in 0.199 s 20/10/07 20:26:51 INFO DAGScheduler: Job 3 finished: isEmpty at DeltaSync.java:384, took 0.204404 s 20/10/07 20:26:52 INFO SparkContext: Starting job: countByKey at SparkHoodieBloomIndex.java:114 20/10/07 20:26:52 INFO DAGScheduler: Registering RDD 20 (mapToPair at SparkWriteHelper.java:54) as input to shuffle 1 20/10/07 20:26:52 INFO DAGScheduler: Registering RDD 24 (countByKey at SparkHoodieBloomIndex.java:114) as input to shuffle 0 20/10/07 20:26:52 INFO DAGScheduler: Got job 4 (countByKey at SparkHoodieBloomIndex.java:114) with 10 output partitions 20/10/07 20:26:52 INFO DAGScheduler: Final stage: ResultStage 6 (countByKey at SparkHoodieBloomIndex.java:114) 20/10/07 20:26:52 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 5) 20/10/07 20:26:52 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 5) 20/10/07 20:26:52 INFO DAGScheduler: Submitting ShuffleMapStage 4 (MapPartitionsRDD[20] at mapToPair at SparkWriteHelper.java:54), which has no missing parents 20/10/07 20:26:52 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 66.4 KB, free 1608.8 MB) 20/10/07 20:26:52 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 24.4 KB, free 1608.7 MB) 20/10/07 20:26:52 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 24.4 KB, free: 1608.9 MB) 20/10/07 20:26:52 INFO SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:52 INFO DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 4 (MapPartitionsRDD[20] at mapToPair at SparkWriteHelper.java:54) (first 15 tasks are for partitions Vector(0, 1)) 20/10/07 20:26:52 INFO YarnClusterScheduler: Adding task set 4.0 with 2 tasks 20/10/07 20:26:52 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 4, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 0, PROCESS_LOCAL, 7872 bytes) 20/10/07 20:26:52 INFO TaskSetManager: Starting task 1.0 in stage 4.0 (TID 5, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 1, PROCESS_LOCAL, 7872 bytes) 20/10/07 20:26:52 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 24.4 KB, free: 5.0 GB) 20/10/07 20:26:53 INFO TaskSetManager: Finished task 1.0 in stage 4.0 (TID 5) in 247 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/2) 20/10/07 20:26:53 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 4) in 248 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (2/2) 20/10/07 20:26:53 INFO YarnClusterScheduler: Removed TaskSet 4.0, whose tasks have all completed, from pool 20/10/07 20:26:53 INFO DAGScheduler: ShuffleMapStage 4 (mapToPair at SparkWriteHelper.java:54) finished in 0.269 s 20/10/07 20:26:53 INFO DAGScheduler: looking for newly runnable stages 20/10/07 20:26:53 INFO DAGScheduler: running: Set() 20/10/07 20:26:53 INFO DAGScheduler: waiting: Set(ShuffleMapStage 5, ResultStage 6) 20/10/07 20:26:53 INFO DAGScheduler: failed: Set() 20/10/07 20:26:53 INFO DAGScheduler: Submitting ShuffleMapStage 5 (MapPartitionsRDD[24] at countByKey at SparkHoodieBloomIndex.java:114), which has no missing parents 20/10/07 20:26:53 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 6.4 KB, free 1608.7 MB) 20/10/07 20:26:53 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 3.5 KB, free 1608.7 MB) 20/10/07 20:26:53 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 3.5 KB, free: 1608.9 MB) 20/10/07 20:26:53 INFO SparkContext: Created broadcast 5 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:53 INFO DAGScheduler: Submitting 10 missing tasks from ShuffleMapStage 5 (MapPartitionsRDD[24] at countByKey at SparkHoodieBloomIndex.java:114) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:26:53 INFO YarnClusterScheduler: Adding task set 5.0 with 10 tasks 20/10/07 20:26:53 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 6, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 0, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 1.0 in stage 5.0 (TID 7, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 1, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 2.0 in stage 5.0 (TID 8, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 2, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 3.0 in stage 5.0 (TID 9, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 3, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 4.0 in stage 5.0 (TID 10, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 5, partition 4, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 5.0 in stage 5.0 (TID 11, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 5, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 6.0 in stage 5.0 (TID 12, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 6, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 7.0 in stage 5.0 (TID 13, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 7, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 8.0 in stage 5.0 (TID 14, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 8, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO TaskSetManager: Starting task 9.0 in stage 5.0 (TID 15, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 5, partition 9, NODE_LOCAL, 7640 bytes) 20/10/07 20:26:53 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:53 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46696 20/10/07 20:26:53 INFO BlockManagerInfo: Added rdd_22_8 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:26:53 INFO BlockManagerInfo: Added rdd_22_3 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 148.0 B, free: 5.0 GB) 20/10/07 20:26:53 INFO TaskSetManager: Finished task 8.0 in stage 5.0 (TID 14) in 556 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/10) 20/10/07 20:26:53 INFO TaskSetManager: Finished task 3.0 in stage 5.0 (TID 9) in 557 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (2/10) 20/10/07 20:26:54 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:54 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:54 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:54 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46694 20/10/07 20:26:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46704 20/10/07 20:26:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46692 20/10/07 20:26:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46698 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_7 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 151.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_2 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 153.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_5 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 293.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_4 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_9 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 (size: 151.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 2.0 in stage 5.0 (TID 8) in 2485 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (3/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 7.0 in stage 5.0 (TID 13) in 2484 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (4/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 5.0 in stage 5.0 (TID 11) in 2488 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (5/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 6) in 2493 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (6/10) 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_6 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added rdd_22_1 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 9.0 in stage 5.0 (TID 15) in 2551 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 5) (7/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 4.0 in stage 5.0 (TID 10) in 2555 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 5) (8/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 6.0 in stage 5.0 (TID 12) in 2657 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (9/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 1.0 in stage 5.0 (TID 7) in 2659 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (10/10) 20/10/07 20:26:55 INFO YarnClusterScheduler: Removed TaskSet 5.0, whose tasks have all completed, from pool 20/10/07 20:26:55 INFO DAGScheduler: ShuffleMapStage 5 (countByKey at SparkHoodieBloomIndex.java:114) finished in 2.674 s 20/10/07 20:26:55 INFO DAGScheduler: looking for newly runnable stages 20/10/07 20:26:55 INFO DAGScheduler: running: Set() 20/10/07 20:26:55 INFO DAGScheduler: waiting: Set(ResultStage 6) 20/10/07 20:26:55 INFO DAGScheduler: failed: Set() 20/10/07 20:26:55 INFO DAGScheduler: Submitting ResultStage 6 (ShuffledRDD[25] at countByKey at SparkHoodieBloomIndex.java:114), which has no missing parents 20/10/07 20:26:55 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 4.0 KB, free 1608.7 MB) 20/10/07 20:26:55 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 2.3 KB, free 1608.7 MB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 2.3 KB, free: 1608.8 MB) 20/10/07 20:26:55 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:55 INFO DAGScheduler: Submitting 10 missing tasks from ResultStage 6 (ShuffledRDD[25] at countByKey at SparkHoodieBloomIndex.java:114) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:26:55 INFO YarnClusterScheduler: Adding task set 6.0 with 10 tasks 20/10/07 20:26:55 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 16, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 0, NODE_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 1.0 in stage 6.0 (TID 17, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 1, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 2.0 in stage 6.0 (TID 18, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 17, partition 2, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 3.0 in stage 6.0 (TID 19, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 12, partition 3, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 4.0 in stage 6.0 (TID 20, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 20, partition 4, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 5.0 in stage 6.0 (TID 21, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 15, partition 5, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 6.0 in stage 6.0 (TID 22, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 6, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 7.0 in stage 6.0 (TID 23, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 16, partition 7, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 8.0 in stage 6.0 (TID 24, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 8, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO TaskSetManager: Starting task 9.0 in stage 6.0 (TID 25, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 9, partition 9, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:26:55 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:55 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.17.203:46698 20/10/07 20:26:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.17.203:46704 20/10/07 20:26:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.17.203:46694 20/10/07 20:26:55 INFO TaskSetManager: Finished task 6.0 in stage 6.0 (TID 22) in 85 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (1/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 8.0 in stage 6.0 (TID 24) in 88 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (2/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 1.0 in stage 6.0 (TID 17) in 91 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (3/10) 20/10/07 20:26:55 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 16) in 119 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (4/10) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_3_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 23.1 KB, free: 1608.9 MB) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_3_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 23.1 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 106 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 107 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 96 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 93 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 110 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 109 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 91 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 90 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 87 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_5_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_5_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 3.5 KB, free: 1608.9 MB) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_5_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 in memory (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_5_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 in memory (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_5_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 in memory (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:44249 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_5_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 in memory (size: 3.5 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 104 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 102 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_4_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 24.4 KB, free: 1608.9 MB) 20/10/07 20:26:56 INFO BlockManagerInfo: Removed broadcast_4_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 24.4 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 89 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 100 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 108 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 103 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 98 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 94 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 99 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 95 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 105 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 97 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 92 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 88 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 101 20/10/07 20:26:56 INFO ContextCleaner: Cleaned accumulator 111 20/10/07 20:26:56 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:39303 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:39987 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:45515 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:46051 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:26:56 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.20.51:47910 20/10/07 20:26:56 INFO TaskSetManager: Finished task 3.0 in stage 6.0 (TID 19) in 973 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 12) (5/10) 20/10/07 20:26:56 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.30.101:59134 20/10/07 20:26:56 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.30.101:59146 20/10/07 20:26:57 INFO TaskSetManager: Finished task 9.0 in stage 6.0 (TID 25) in 1179 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 9) (6/10) 20/10/07 20:26:57 INFO TaskSetManager: Finished task 2.0 in stage 6.0 (TID 18) in 1201 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 17) (7/10) 20/10/07 20:26:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.19.77:48120 20/10/07 20:26:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.19.77:48116 20/10/07 20:26:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 172.31.19.77:48118 20/10/07 20:26:57 INFO TaskSetManager: Finished task 4.0 in stage 6.0 (TID 20) in 1605 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 20) (8/10) 20/10/07 20:26:57 INFO TaskSetManager: Finished task 7.0 in stage 6.0 (TID 23) in 1645 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 16) (9/10) 20/10/07 20:26:57 INFO TaskSetManager: Finished task 5.0 in stage 6.0 (TID 21) in 1654 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 15) (10/10) 20/10/07 20:26:57 INFO YarnClusterScheduler: Removed TaskSet 6.0, whose tasks have all completed, from pool 20/10/07 20:26:57 INFO DAGScheduler: ResultStage 6 (countByKey at SparkHoodieBloomIndex.java:114) finished in 1.668 s 20/10/07 20:26:57 INFO DAGScheduler: Job 4 finished: countByKey at SparkHoodieBloomIndex.java:114, took 4.659268 s 20/10/07 20:26:57 INFO SparkContext: Starting job: collect at HoodieSparkEngineContext.java:74 20/10/07 20:26:57 INFO DAGScheduler: Got job 5 (collect at HoodieSparkEngineContext.java:74) with 1 output partitions 20/10/07 20:26:57 INFO DAGScheduler: Final stage: ResultStage 7 (collect at HoodieSparkEngineContext.java:74) 20/10/07 20:26:57 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:26:57 INFO DAGScheduler: Missing parents: List() 20/10/07 20:26:57 INFO DAGScheduler: Submitting ResultStage 7 (MapPartitionsRDD[27] at flatMap at HoodieSparkEngineContext.java:74), which has no missing parents 20/10/07 20:26:57 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 262.9 KB, free 1608.6 MB) 20/10/07 20:26:57 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 79.8 KB, free 1608.6 MB) 20/10/07 20:26:57 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 79.8 KB, free: 1608.8 MB) 20/10/07 20:26:57 INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:57 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 7 (MapPartitionsRDD[27] at flatMap at HoodieSparkEngineContext.java:74) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:26:57 INFO YarnClusterScheduler: Adding task set 7.0 with 1 tasks 20/10/07 20:26:57 INFO TaskSetManager: Starting task 0.0 in stage 7.0 (TID 26, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 3, partition 0, PROCESS_LOCAL, 7713 bytes) 20/10/07 20:26:58 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:45015 (size: 79.8 KB, free: 5.0 GB) 20/10/07 20:26:58 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID 26) in 1022 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 3) (1/1) 20/10/07 20:26:58 INFO YarnClusterScheduler: Removed TaskSet 7.0, whose tasks have all completed, from pool 20/10/07 20:26:58 INFO DAGScheduler: ResultStage 7 (collect at HoodieSparkEngineContext.java:74) finished in 1.063 s 20/10/07 20:26:58 INFO DAGScheduler: Job 5 finished: collect at HoodieSparkEngineContext.java:74, took 1.066932 s 20/10/07 20:26:58 INFO SparkContext: Starting job: collect at HoodieSparkEngineContext.java:69 20/10/07 20:26:58 INFO DAGScheduler: Got job 6 (collect at HoodieSparkEngineContext.java:69) with 1 output partitions 20/10/07 20:26:58 INFO DAGScheduler: Final stage: ResultStage 8 (collect at HoodieSparkEngineContext.java:69) 20/10/07 20:26:58 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:26:58 INFO DAGScheduler: Missing parents: List() 20/10/07 20:26:58 INFO DAGScheduler: Submitting ResultStage 8 (MapPartitionsRDD[29] at map at HoodieSparkEngineContext.java:69), which has no missing parents 20/10/07 20:26:58 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 262.7 KB, free 1608.3 MB) 20/10/07 20:26:58 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 79.7 KB, free 1608.2 MB) 20/10/07 20:26:58 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 79.7 KB, free: 1608.7 MB) 20/10/07 20:26:58 INFO SparkContext: Created broadcast 8 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:58 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 8 (MapPartitionsRDD[29] at map at HoodieSparkEngineContext.java:69) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:26:58 INFO YarnClusterScheduler: Adding task set 8.0 with 1 tasks 20/10/07 20:26:58 INFO TaskSetManager: Starting task 0.0 in stage 8.0 (TID 27, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 7, partition 0, PROCESS_LOCAL, 7710 bytes) 20/10/07 20:26:59 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 (size: 79.7 KB, free: 5.0 GB) 20/10/07 20:26:59 INFO TaskSetManager: Finished task 0.0 in stage 8.0 (TID 27) in 1020 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 7) (1/1) 20/10/07 20:26:59 INFO YarnClusterScheduler: Removed TaskSet 8.0, whose tasks have all completed, from pool 20/10/07 20:26:59 INFO DAGScheduler: ResultStage 8 (collect at HoodieSparkEngineContext.java:69) finished in 1.059 s 20/10/07 20:26:59 INFO DAGScheduler: Job 6 finished: collect at HoodieSparkEngineContext.java:69, took 1.060988 s 20/10/07 20:26:59 INFO SparkContext: Starting job: countByKey at SparkHoodieBloomIndex.java:147 20/10/07 20:26:59 INFO DAGScheduler: Registering RDD 33 (countByKey at SparkHoodieBloomIndex.java:147) as input to shuffle 2 20/10/07 20:26:59 INFO DAGScheduler: Got job 7 (countByKey at SparkHoodieBloomIndex.java:147) with 10 output partitions 20/10/07 20:26:59 INFO DAGScheduler: Final stage: ResultStage 11 (countByKey at SparkHoodieBloomIndex.java:147) 20/10/07 20:26:59 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 10) 20/10/07 20:26:59 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 10) 20/10/07 20:26:59 INFO DAGScheduler: Submitting ShuffleMapStage 10 (MapPartitionsRDD[33] at countByKey at SparkHoodieBloomIndex.java:147), which has no missing parents 20/10/07 20:26:59 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 7.7 KB, free 1608.2 MB) 20/10/07 20:26:59 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 4.0 KB, free 1608.2 MB) 20/10/07 20:26:59 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 4.0 KB, free: 1608.7 MB) 20/10/07 20:26:59 INFO SparkContext: Created broadcast 9 from broadcast at DAGScheduler.scala:1203 20/10/07 20:26:59 INFO DAGScheduler: Submitting 10 missing tasks from ShuffleMapStage 10 (MapPartitionsRDD[33] at countByKey at SparkHoodieBloomIndex.java:147) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:26:59 INFO YarnClusterScheduler: Adding task set 10.0 with 10 tasks 20/10/07 20:26:59 INFO TaskSetManager: Starting task 4.0 in stage 10.0 (TID 28, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 5, partition 4, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 0.0 in stage 10.0 (TID 29, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 0, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 1.0 in stage 10.0 (TID 30, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 1, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 2.0 in stage 10.0 (TID 31, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 2, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 3.0 in stage 10.0 (TID 32, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 3, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 9.0 in stage 10.0 (TID 33, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 5, partition 9, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 5.0 in stage 10.0 (TID 34, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 5, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 6.0 in stage 10.0 (TID 35, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 6, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 7.0 in stage 10.0 (TID 36, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 7, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO TaskSetManager: Starting task 8.0 in stage 10.0 (TID 37, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 8, PROCESS_LOCAL, 7640 bytes) 20/10/07 20:26:59 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:26:59 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:26:59 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:26:59 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:26:59 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 6.0 in stage 10.0 (TID 35) in 108 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (1/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 1.0 in stage 10.0 (TID 30) in 118 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (2/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 8.0 in stage 10.0 (TID 37) in 129 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (3/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 2.0 in stage 10.0 (TID 31) in 135 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (4/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 3.0 in stage 10.0 (TID 32) in 136 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (5/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 0.0 in stage 10.0 (TID 29) in 144 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (6/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 4.0 in stage 10.0 (TID 28) in 148 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 5) (7/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 7.0 in stage 10.0 (TID 36) in 147 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (8/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 9.0 in stage 10.0 (TID 33) in 157 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 5) (9/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 5.0 in stage 10.0 (TID 34) in 165 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (10/10) 20/10/07 20:27:00 INFO YarnClusterScheduler: Removed TaskSet 10.0, whose tasks have all completed, from pool 20/10/07 20:27:00 INFO DAGScheduler: ShuffleMapStage 10 (countByKey at SparkHoodieBloomIndex.java:147) finished in 0.183 s 20/10/07 20:27:00 INFO DAGScheduler: looking for newly runnable stages 20/10/07 20:27:00 INFO DAGScheduler: running: Set() 20/10/07 20:27:00 INFO DAGScheduler: waiting: Set(ResultStage 11) 20/10/07 20:27:00 INFO DAGScheduler: failed: Set() 20/10/07 20:27:00 INFO DAGScheduler: Submitting ResultStage 11 (ShuffledRDD[34] at countByKey at SparkHoodieBloomIndex.java:147), which has no missing parents 20/10/07 20:27:00 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 4.0 KB, free 1608.2 MB) 20/10/07 20:27:00 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 2.3 KB, free 1608.2 MB) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 2.3 KB, free: 1608.7 MB) 20/10/07 20:27:00 INFO SparkContext: Created broadcast 10 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:00 INFO DAGScheduler: Submitting 10 missing tasks from ResultStage 11 (ShuffledRDD[34] at countByKey at SparkHoodieBloomIndex.java:147) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:27:00 INFO YarnClusterScheduler: Adding task set 11.0 with 10 tasks 20/10/07 20:27:00 INFO TaskSetManager: Starting task 0.0 in stage 11.0 (TID 38, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 16, partition 0, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 1.0 in stage 11.0 (TID 39, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 10, partition 1, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 2.0 in stage 11.0 (TID 40, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 11, partition 2, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 3.0 in stage 11.0 (TID 41, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 3, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 4.0 in stage 11.0 (TID 42, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 4, partition 4, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 5.0 in stage 11.0 (TID 43, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 5, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 6.0 in stage 11.0 (TID 44, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 18, partition 6, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 7.0 in stage 11.0 (TID 45, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 3, partition 7, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 8.0 in stage 11.0 (TID 46, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 7, partition 8, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO TaskSetManager: Starting task 9.0 in stage 11.0 (TID 47, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 15, partition 9, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.17.203:46698 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.17.203:46696 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:45015 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:45515 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.19.77:48116 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.19.77:48118 20/10/07 20:27:00 INFO TaskSetManager: Finished task 5.0 in stage 11.0 (TID 43) in 59 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 3.0 in stage 11.0 (TID 41) in 64 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (2/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 9.0 in stage 11.0 (TID 47) in 88 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 15) (3/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 0.0 in stage 11.0 (TID 38) in 92 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 16) (4/10) 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.19.77:48104 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.20.51:47904 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.20.51:47906 20/10/07 20:27:00 INFO TaskSetManager: Finished task 8.0 in stage 11.0 (TID 46) in 180 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 7) (5/10) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 4.0 in stage 11.0 (TID 42) in 289 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 4) (6/10) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:37747 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO TaskSetManager: Finished task 7.0 in stage 11.0 (TID 45) in 329 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 3) (7/10) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:37377 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.30.101:59132 20/10/07 20:27:00 INFO TaskSetManager: Finished task 1.0 in stage 11.0 (TID 39) in 885 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 10) (8/10) 20/10/07 20:27:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.20.51:47908 20/10/07 20:27:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 172.31.20.51:47918 20/10/07 20:27:01 INFO TaskSetManager: Finished task 2.0 in stage 11.0 (TID 40) in 1389 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 11) (9/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 6.0 in stage 11.0 (TID 44) in 1395 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 18) (10/10) 20/10/07 20:27:01 INFO YarnClusterScheduler: Removed TaskSet 11.0, whose tasks have all completed, from pool 20/10/07 20:27:01 INFO DAGScheduler: ResultStage 11 (countByKey at SparkHoodieBloomIndex.java:147) finished in 1.405 s 20/10/07 20:27:01 INFO DAGScheduler: Job 7 finished: countByKey at SparkHoodieBloomIndex.java:147, took 1.595669 s 20/10/07 20:27:01 INFO MapPartitionsRDD: Removing RDD 22 from persistence list 20/10/07 20:27:01 INFO BlockManager: Removing RDD 22 20/10/07 20:27:01 INFO MapPartitionsRDD: Removing RDD 43 from persistence list 20/10/07 20:27:01 INFO BlockManager: Removing RDD 43 20/10/07 20:27:01 INFO SparkContext: Starting job: countByKey at BaseSparkCommitActionExecutor.java:133 20/10/07 20:27:01 INFO DAGScheduler: Registering RDD 37 (mapToPair at SparkHoodieBloomIndex.java:265) as input to shuffle 6 20/10/07 20:27:01 INFO DAGScheduler: Registering RDD 43 (flatMapToPair at SparkHoodieBloomIndex.java:273) as input to shuffle 4 20/10/07 20:27:01 INFO DAGScheduler: Registering RDD 44 (mapToPair at SparkHoodieBloomIndex.java:286) as input to shuffle 3 20/10/07 20:27:01 INFO DAGScheduler: Registering RDD 52 (countByKey at BaseSparkCommitActionExecutor.java:133) as input to shuffle 5 20/10/07 20:27:01 INFO DAGScheduler: Got job 8 (countByKey at BaseSparkCommitActionExecutor.java:133) with 10 output partitions 20/10/07 20:27:01 INFO DAGScheduler: Final stage: ResultStage 17 (countByKey at BaseSparkCommitActionExecutor.java:133) 20/10/07 20:27:01 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 16) 20/10/07 20:27:01 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 16) 20/10/07 20:27:01 INFO DAGScheduler: Submitting ShuffleMapStage 15 (MapPartitionsRDD[44] at mapToPair at SparkHoodieBloomIndex.java:286), which has no missing parents 20/10/07 20:27:01 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 6.1 KB, free 1608.2 MB) 20/10/07 20:27:01 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 3.4 KB, free 1608.2 MB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 3.4 KB, free: 1608.7 MB) 20/10/07 20:27:01 INFO SparkContext: Created broadcast 11 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:01 INFO DAGScheduler: Submitting 10 missing tasks from ShuffleMapStage 15 (MapPartitionsRDD[44] at mapToPair at SparkHoodieBloomIndex.java:286) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:27:01 INFO YarnClusterScheduler: Adding task set 15.0 with 10 tasks 20/10/07 20:27:01 INFO TaskSetManager: Starting task 0.0 in stage 15.0 (TID 48, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 0, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 1.0 in stage 15.0 (TID 49, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 1, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 2.0 in stage 15.0 (TID 50, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 2, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 3.0 in stage 15.0 (TID 51, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 5, partition 3, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 4.0 in stage 15.0 (TID 52, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 4, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 5.0 in stage 15.0 (TID 53, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 5, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 6.0 in stage 15.0 (TID 54, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 14, partition 6, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 7.0 in stage 15.0 (TID 55, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 7, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 8.0 in stage 15.0 (TID 56, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 5, partition 8, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 9.0 in stage 15.0 (TID 57, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 9, NODE_LOCAL, 7640 bytes) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46698 20/10/07 20:27:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46696 20/10/07 20:27:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46694 20/10/07 20:27:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46704 20/10/07 20:27:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 172.31.17.203:46692 20/10/07 20:27:01 INFO TaskSetManager: Finished task 7.0 in stage 15.0 (TID 55) in 122 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 4.0 in stage 15.0 (TID 52) in 179 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (2/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 9.0 in stage 15.0 (TID 57) in 193 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (3/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 3.0 in stage 15.0 (TID 51) in 195 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 5) (4/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 8.0 in stage 15.0 (TID 56) in 195 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 5) (5/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 2.0 in stage 15.0 (TID 50) in 198 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (6/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 5.0 in stage 15.0 (TID 53) in 201 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (7/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 0.0 in stage 15.0 (TID 48) in 203 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (8/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 1.0 in stage 15.0 (TID 49) in 206 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (9/10) 20/10/07 20:27:01 INFO TaskSetManager: Finished task 6.0 in stage 15.0 (TID 54) in 221 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 14) (10/10) 20/10/07 20:27:01 INFO YarnClusterScheduler: Removed TaskSet 15.0, whose tasks have all completed, from pool 20/10/07 20:27:01 INFO DAGScheduler: ShuffleMapStage 15 (mapToPair at SparkHoodieBloomIndex.java:286) finished in 0.239 s 20/10/07 20:27:01 INFO DAGScheduler: looking for newly runnable stages 20/10/07 20:27:01 INFO DAGScheduler: running: Set() 20/10/07 20:27:01 INFO DAGScheduler: waiting: Set(ShuffleMapStage 16, ResultStage 17) 20/10/07 20:27:01 INFO DAGScheduler: failed: Set() 20/10/07 20:27:01 INFO DAGScheduler: Submitting ShuffleMapStage 16 (MapPartitionsRDD[52] at countByKey at BaseSparkCommitActionExecutor.java:133), which has no missing parents 20/10/07 20:27:01 INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 7.2 KB, free 1608.2 MB) 20/10/07 20:27:01 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 3.9 KB, free 1608.2 MB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 3.9 KB, free: 1608.7 MB) 20/10/07 20:27:01 INFO SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:01 INFO DAGScheduler: Submitting 10 missing tasks from ShuffleMapStage 16 (MapPartitionsRDD[52] at countByKey at BaseSparkCommitActionExecutor.java:133) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:27:01 INFO YarnClusterScheduler: Adding task set 16.0 with 10 tasks 20/10/07 20:27:01 INFO TaskSetManager: Starting task 0.0 in stage 16.0 (TID 58, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 12, partition 0, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 1.0 in stage 16.0 (TID 59, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 4, partition 1, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 2.0 in stage 16.0 (TID 60, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 11, partition 2, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 3.0 in stage 16.0 (TID 61, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 10, partition 3, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 4.0 in stage 16.0 (TID 62, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 20, partition 4, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 5.0 in stage 16.0 (TID 63, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 7, partition 5, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 6.0 in stage 16.0 (TID 64, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 1, partition 6, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 7.0 in stage 16.0 (TID 65, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 7, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 8.0 in stage 16.0 (TID 66, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 15, partition 8, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO TaskSetManager: Starting task 9.0 in stage 16.0 (TID 67, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 9, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:45515 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:44249 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:46051 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:37747 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:01 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.17.203:46704 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.17.203:46694 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.17.203:46704 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.17.203:46694 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.19.77:48104 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.30.101:59132 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_9 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 151.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.20.51:47904 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_7 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 151.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.19.77:48104 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.20.51:47910 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.20.51:47908 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.30.101:59132 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.20.51:47904 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.19.77:48118 20/10/07 20:27:02 INFO TaskSetManager: Finished task 7.0 in stage 16.0 (TID 65) in 182 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (1/10) 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.19.77:48120 20/10/07 20:27:02 INFO TaskSetManager: Finished task 9.0 in stage 16.0 (TID 67) in 187 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (2/10) 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.20.51:47910 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.19.77:48118 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.19.77:48120 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_5 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_3 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:37747 (size: 148.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.20.51:47908 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_8 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:45515 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_1 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:44249 (size: 293.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_4 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:46051 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO TaskSetManager: Finished task 5.0 in stage 16.0 (TID 63) in 331 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 7) (3/10) 20/10/07 20:27:02 INFO TaskSetManager: Finished task 3.0 in stage 16.0 (TID 61) in 343 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 10) (4/10) 20/10/07 20:27:02 INFO TaskSetManager: Finished task 8.0 in stage 16.0 (TID 66) in 373 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 15) (5/10) 20/10/07 20:27:02 INFO BlockManagerInfo: Added rdd_50_2 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 (size: 153.0 B, free: 5.0 GB) 20/10/07 20:27:02 INFO TaskSetManager: Finished task 4.0 in stage 16.0 (TID 62) in 396 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 20) (6/10) 20/10/07 20:27:02 INFO TaskSetManager: Finished task 1.0 in stage 16.0 (TID 59) in 409 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 4) (7/10) 20/10/07 20:27:02 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:32963 (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:02 INFO TaskSetManager: Finished task 0.0 in stage 16.0 (TID 58) in 415 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 12) (8/10) 20/10/07 20:27:02 INFO TaskSetManager: Finished task 2.0 in stage 16.0 (TID 60) in 482 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 11) (9/10) 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 172.31.30.101:59128 20/10/07 20:27:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 172.31.30.101:59128 20/10/07 20:27:03 INFO BlockManagerInfo: Added rdd_50_6 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:32963 (size: 302.0 B, free: 5.0 GB) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 6.0 in stage 16.0 (TID 64) in 1221 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 1) (10/10) 20/10/07 20:27:03 INFO YarnClusterScheduler: Removed TaskSet 16.0, whose tasks have all completed, from pool 20/10/07 20:27:03 INFO DAGScheduler: ShuffleMapStage 16 (countByKey at BaseSparkCommitActionExecutor.java:133) finished in 1.236 s 20/10/07 20:27:03 INFO DAGScheduler: looking for newly runnable stages 20/10/07 20:27:03 INFO DAGScheduler: running: Set() 20/10/07 20:27:03 INFO DAGScheduler: waiting: Set(ResultStage 17) 20/10/07 20:27:03 INFO DAGScheduler: failed: Set() 20/10/07 20:27:03 INFO DAGScheduler: Submitting ResultStage 17 (ShuffledRDD[53] at countByKey at BaseSparkCommitActionExecutor.java:133), which has no missing parents 20/10/07 20:27:03 INFO MemoryStore: Block broadcast_13 stored as values in memory (estimated size 4.0 KB, free 1608.2 MB) 20/10/07 20:27:03 INFO MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 2.3 KB, free 1608.2 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 2.3 KB, free: 1608.7 MB) 20/10/07 20:27:03 INFO SparkContext: Created broadcast 13 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:03 INFO DAGScheduler: Submitting 10 missing tasks from ResultStage 17 (ShuffledRDD[53] at countByKey at BaseSparkCommitActionExecutor.java:133) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:27:03 INFO YarnClusterScheduler: Adding task set 17.0 with 10 tasks 20/10/07 20:27:03 INFO TaskSetManager: Starting task 0.0 in stage 17.0 (TID 68, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 7, partition 0, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 1.0 in stage 17.0 (TID 69, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 9, partition 1, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 2.0 in stage 17.0 (TID 70, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 2, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 3.0 in stage 17.0 (TID 71, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 4, partition 3, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 4.0 in stage 17.0 (TID 72, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 17, partition 4, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 5.0 in stage 17.0 (TID 73, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 18, partition 5, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 6.0 in stage 17.0 (TID 74, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 16, partition 6, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 7.0 in stage 17.0 (TID 75, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 20, partition 7, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 8.0 in stage 17.0 (TID 76, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 8, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO TaskSetManager: Starting task 9.0 in stage 17.0 (TID 77, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 11, partition 9, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.17.203:46696 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.17.203:46694 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:46051 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.19.77:48116 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:39987 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:39303 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:37377 (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.20.51:47904 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.19.77:48120 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.20.51:47908 20/10/07 20:27:03 INFO TaskSetManager: Finished task 8.0 in stage 17.0 (TID 76) in 40 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 2.0 in stage 17.0 (TID 70) in 42 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (2/10) 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.19.77:48104 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.30.101:59146 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.30.101:59134 20/10/07 20:27:03 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 172.31.20.51:47918 20/10/07 20:27:03 INFO TaskSetManager: Finished task 6.0 in stage 17.0 (TID 74) in 52 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 16) (3/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 3.0 in stage 17.0 (TID 71) in 62 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 4) (4/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 9.0 in stage 17.0 (TID 77) in 62 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 11) (5/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 0.0 in stage 17.0 (TID 68) in 65 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 7) (6/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 7.0 in stage 17.0 (TID 75) in 64 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 20) (7/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 4.0 in stage 17.0 (TID 72) in 73 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 17) (8/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 1.0 in stage 17.0 (TID 69) in 74 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 9) (9/10) 20/10/07 20:27:03 INFO TaskSetManager: Finished task 5.0 in stage 17.0 (TID 73) in 152 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 18) (10/10) 20/10/07 20:27:03 INFO YarnClusterScheduler: Removed TaskSet 17.0, whose tasks have all completed, from pool 20/10/07 20:27:03 INFO DAGScheduler: ResultStage 17 (countByKey at BaseSparkCommitActionExecutor.java:133) finished in 0.162 s 20/10/07 20:27:03 INFO DAGScheduler: Job 8 finished: countByKey at BaseSparkCommitActionExecutor.java:133, took 1.645656 s 20/10/07 20:27:03 INFO SparkContext: Starting job: collectAsMap at UpsertPartitioner.java:221 20/10/07 20:27:03 INFO DAGScheduler: Got job 9 (collectAsMap at UpsertPartitioner.java:221) with 1 output partitions 20/10/07 20:27:03 INFO DAGScheduler: Final stage: ResultStage 18 (collectAsMap at UpsertPartitioner.java:221) 20/10/07 20:27:03 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:27:03 INFO DAGScheduler: Missing parents: List() 20/10/07 20:27:03 INFO DAGScheduler: Submitting ResultStage 18 (MapPartitionsRDD[55] at mapToPair at UpsertPartitioner.java:220), which has no missing parents 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 247 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 318 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 234 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 275 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 263 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 330 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 284 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 321 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 225 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 305 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 250 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 287 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 317 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 301 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 319 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 230 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 267 20/10/07 20:27:03 INFO MemoryStore: Block broadcast_14 stored as values in memory (estimated size 263.3 KB, free 1607.9 MB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 226 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 303 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 323 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 188 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 291 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 221 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 231 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 191 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 335 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 211 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 216 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 239 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 359 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 272 20/10/07 20:27:03 INFO MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 80.1 KB, free 1607.8 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 80.1 KB, free: 1608.6 MB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned shuffle 2 20/10/07 20:27:03 INFO SparkContext: Created broadcast 14 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 270 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 289 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 251 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 348 20/10/07 20:27:03 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 18 (MapPartitionsRDD[55] at mapToPair at UpsertPartitioner.java:220) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:27:03 INFO YarnClusterScheduler: Adding task set 18.0 with 1 tasks 20/10/07 20:27:03 INFO TaskSetManager: Starting task 0.0 in stage 18.0 (TID 78, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 18, partition 0, PROCESS_LOCAL, 7713 bytes) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 3.9 KB, free: 1608.7 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:46051 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-30-101.ap-southeast-2.compute.internal:32963 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:44249 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-30-101.ap-southeast-2.compute.internal:37747 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:45515 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_12_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 in memory (size: 3.9 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 241 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 229 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 288 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 265 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 271 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 255 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 281 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 351 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 327 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 237 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 340 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 199 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 308 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 274 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 205 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 279 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 349 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 332 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 326 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 304 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 358 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 244 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 187 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 338 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 299 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_9_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_9_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 in memory (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:37377 (size: 80.1 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_9_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 in memory (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_9_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 in memory (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_9_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 4.0 KB, free: 1608.7 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_9_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 in memory (size: 4.0 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 214 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 341 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 212 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 224 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 192 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 190 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 253 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 204 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 312 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 246 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 280 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 269 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 208 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 257 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 232 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 252 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 193 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 223 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 328 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 218 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 264 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 201 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 286 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 311 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_8_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 79.7 KB, free: 1608.7 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_8_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 in memory (size: 79.7 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 342 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 256 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:45515 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 2.3 KB, free: 1608.7 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-30-101.ap-southeast-2.compute.internal:37747 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:37377 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_10_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:45015 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 360 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 313 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 294 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 314 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 307 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 295 20/10/07 20:27:03 INFO ContextCleaner: Cleaned shuffle 5 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 310 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 357 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 200 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 222 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 245 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 258 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 356 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 334 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 227 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 260 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 320 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 283 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 209 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 249 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 228 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 195 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_11_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 in memory (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_11_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 in memory (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_11_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:43295 in memory (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_11_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 3.4 KB, free: 1608.7 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_11_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:45915 in memory (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_11_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 3.4 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 298 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 316 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 220 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 336 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 277 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 261 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 197 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 266 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 355 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 282 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 302 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 343 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 262 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 353 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 293 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 202 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 344 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 346 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 309 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 217 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 300 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 219 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 352 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 259 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 296 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 285 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 361 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 210 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 242 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 322 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 290 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 254 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:46051 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 2.3 KB, free: 1608.7 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:37377 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-30-101.ap-southeast-2.compute.internal:39987 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-30-101.ap-southeast-2.compute.internal:39303 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_13_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 in memory (size: 2.3 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 240 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 196 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 354 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 236 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 194 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 306 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 268 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 238 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 347 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 297 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 337 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 278 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 207 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 233 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 292 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 325 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 248 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 345 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 331 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 350 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 315 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 324 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_7_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 in memory (size: 79.8 KB, free: 1608.8 MB) 20/10/07 20:27:03 INFO BlockManagerInfo: Removed broadcast_7_piece0 on ip-172-31-20-51.ap-southeast-2.compute.internal:45015 in memory (size: 79.8 KB, free: 5.0 GB) 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 243 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 206 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 329 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 333 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 213 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 198 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 215 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 276 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 235 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 203 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 273 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 339 20/10/07 20:27:03 INFO ContextCleaner: Cleaned accumulator 189 20/10/07 20:27:03 INFO TaskSetManager: Finished task 0.0 in stage 18.0 (TID 78) in 282 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 18) (1/1) 20/10/07 20:27:03 INFO YarnClusterScheduler: Removed TaskSet 18.0, whose tasks have all completed, from pool 20/10/07 20:27:03 INFO DAGScheduler: ResultStage 18 (collectAsMap at UpsertPartitioner.java:221) finished in 0.361 s 20/10/07 20:27:03 INFO DAGScheduler: Job 9 finished: collectAsMap at UpsertPartitioner.java:221, took 0.363246 s 20/10/07 20:27:04 INFO SparkContext: Starting job: sum at DeltaSync.java:405 20/10/07 20:27:04 INFO DAGScheduler: Registering RDD 56 (mapToPair at BaseSparkCommitActionExecutor.java:167) as input to shuffle 7 20/10/07 20:27:04 INFO DAGScheduler: Got job 10 (sum at DeltaSync.java:405) with 1 output partitions 20/10/07 20:27:04 INFO DAGScheduler: Final stage: ResultStage 24 (sum at DeltaSync.java:405) 20/10/07 20:27:04 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 23) 20/10/07 20:27:04 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 23) 20/10/07 20:27:04 INFO DAGScheduler: Submitting ShuffleMapStage 23 (MapPartitionsRDD[56] at mapToPair at BaseSparkCommitActionExecutor.java:167), which has no missing parents 20/10/07 20:27:04 INFO MemoryStore: Block broadcast_15 stored as values in memory (estimated size 267.2 KB, free 1608.3 MB) 20/10/07 20:27:04 INFO MemoryStore: Block broadcast_15_piece0 stored as bytes in memory (estimated size 81.7 KB, free 1608.2 MB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 81.7 KB, free: 1608.7 MB) 20/10/07 20:27:04 INFO SparkContext: Created broadcast 15 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:04 INFO DAGScheduler: Submitting 10 missing tasks from ShuffleMapStage 23 (MapPartitionsRDD[56] at mapToPair at BaseSparkCommitActionExecutor.java:167) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 20/10/07 20:27:04 INFO YarnClusterScheduler: Adding task set 23.0 with 10 tasks 20/10/07 20:27:04 INFO TaskSetManager: Starting task 3.0 in stage 23.0 (TID 79, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 10, partition 3, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 9.0 in stage 23.0 (TID 80, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 13, partition 9, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 7.0 in stage 23.0 (TID 81, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 19, partition 7, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 6.0 in stage 23.0 (TID 82, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 1, partition 6, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 2.0 in stage 23.0 (TID 83, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 11, partition 2, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 5.0 in stage 23.0 (TID 84, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 7, partition 5, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 4.0 in stage 23.0 (TID 85, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 20, partition 4, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 1.0 in stage 23.0 (TID 86, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 4, partition 1, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 0.0 in stage 23.0 (TID 87, ip-172-31-20-51.ap-southeast-2.compute.internal, executor 12, partition 0, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO TaskSetManager: Starting task 8.0 in stage 23.0 (TID 88, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 15, partition 8, PROCESS_LOCAL, 7730 bytes) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:46075 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:45515 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:39405 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:44393 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:34719 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:46051 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:32963 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:39129 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:44249 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:37747 (size: 81.7 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 5.0 in stage 23.0 (TID 84) in 120 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 7) (1/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 9.0 in stage 23.0 (TID 80) in 205 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 13) (2/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 6.0 in stage 23.0 (TID 82) in 215 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 1) (3/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 7.0 in stage 23.0 (TID 81) in 237 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 19) (4/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 8.0 in stage 23.0 (TID 88) in 240 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 15) (5/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 3.0 in stage 23.0 (TID 79) in 263 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 10) (6/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 4.0 in stage 23.0 (TID 85) in 265 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 20) (7/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 1.0 in stage 23.0 (TID 86) in 269 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 4) (8/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 2.0 in stage 23.0 (TID 83) in 277 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 11) (9/10) 20/10/07 20:27:04 INFO TaskSetManager: Finished task 0.0 in stage 23.0 (TID 87) in 287 ms on ip-172-31-20-51.ap-southeast-2.compute.internal (executor 12) (10/10) 20/10/07 20:27:04 INFO YarnClusterScheduler: Removed TaskSet 23.0, whose tasks have all completed, from pool 20/10/07 20:27:04 INFO DAGScheduler: ShuffleMapStage 23 (mapToPair at BaseSparkCommitActionExecutor.java:167) finished in 0.329 s 20/10/07 20:27:04 INFO DAGScheduler: looking for newly runnable stages 20/10/07 20:27:04 INFO DAGScheduler: running: Set() 20/10/07 20:27:04 INFO DAGScheduler: waiting: Set(ResultStage 24) 20/10/07 20:27:04 INFO DAGScheduler: failed: Set() 20/10/07 20:27:04 INFO DAGScheduler: Submitting ResultStage 24 (MapPartitionsRDD[61] at mapToDouble at DeltaSync.java:405), which has no missing parents 20/10/07 20:27:04 INFO MemoryStore: Block broadcast_16 stored as values in memory (estimated size 328.3 KB, free 1607.9 MB) 20/10/07 20:27:04 INFO MemoryStore: Block broadcast_16_piece0 stored as bytes in memory (estimated size 103.0 KB, free 1607.8 MB) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 103.0 KB, free: 1608.6 MB) 20/10/07 20:27:04 INFO SparkContext: Created broadcast 16 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:04 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 24 (MapPartitionsRDD[61] at mapToDouble at DeltaSync.java:405) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:27:04 INFO YarnClusterScheduler: Adding task set 24.0 with 1 tasks 20/10/07 20:27:04 INFO TaskSetManager: Starting task 0.0 in stage 24.0 (TID 89, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 16, partition 0, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:04 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 (size: 103.0 KB, free: 5.0 GB) 20/10/07 20:27:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 7 to 172.31.19.77:48116 20/10/07 20:27:07 INFO BlockManagerInfo: Added rdd_60_0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 (size: 2.7 KB, free: 5.0 GB) 20/10/07 20:27:07 INFO TaskSetManager: Finished task 0.0 in stage 24.0 (TID 89) in 2798 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 16) (1/1) 20/10/07 20:27:07 INFO YarnClusterScheduler: Removed TaskSet 24.0, whose tasks have all completed, from pool 20/10/07 20:27:07 INFO DAGScheduler: ResultStage 24 (sum at DeltaSync.java:405) finished in 2.838 s 20/10/07 20:27:07 INFO DAGScheduler: Job 10 finished: sum at DeltaSync.java:405, took 3.173911 s 20/10/07 20:27:07 INFO SparkContext: Starting job: sum at DeltaSync.java:406 20/10/07 20:27:07 INFO DAGScheduler: Got job 11 (sum at DeltaSync.java:406) with 1 output partitions 20/10/07 20:27:07 INFO DAGScheduler: Final stage: ResultStage 30 (sum at DeltaSync.java:406) 20/10/07 20:27:07 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 29) 20/10/07 20:27:07 INFO DAGScheduler: Missing parents: List() 20/10/07 20:27:07 INFO DAGScheduler: Submitting ResultStage 30 (MapPartitionsRDD[63] at mapToDouble at DeltaSync.java:406), which has no missing parents 20/10/07 20:27:07 INFO MemoryStore: Block broadcast_17 stored as values in memory (estimated size 328.3 KB, free 1607.5 MB) 20/10/07 20:27:07 INFO MemoryStore: Block broadcast_17_piece0 stored as bytes in memory (estimated size 103.0 KB, free 1607.4 MB) 20/10/07 20:27:07 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 103.0 KB, free: 1608.5 MB) 20/10/07 20:27:07 INFO SparkContext: Created broadcast 17 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:07 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 30 (MapPartitionsRDD[63] at mapToDouble at DeltaSync.java:406) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:27:07 INFO YarnClusterScheduler: Adding task set 30.0 with 1 tasks 20/10/07 20:27:07 INFO TaskSetManager: Starting task 0.0 in stage 30.0 (TID 90, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 16, partition 0, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:07 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 (size: 103.0 KB, free: 5.0 GB) 20/10/07 20:27:07 INFO TaskSetManager: Finished task 0.0 in stage 30.0 (TID 90) in 69 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 16) (1/1) 20/10/07 20:27:07 INFO YarnClusterScheduler: Removed TaskSet 30.0, whose tasks have all completed, from pool 20/10/07 20:27:07 INFO DAGScheduler: ResultStage 30 (sum at DeltaSync.java:406) finished in 0.106 s 20/10/07 20:27:07 INFO DAGScheduler: Job 11 finished: sum at DeltaSync.java:406, took 0.108824 s 20/10/07 20:27:07 ERROR DeltaSync: Delta Sync found errors when writing. Errors/Total=16/16 20/10/07 20:27:07 ERROR DeltaSync: Printing out the top 100 errors 20/10/07 20:27:07 INFO SparkContext: Starting job: take at DeltaSync.java:441 20/10/07 20:27:07 INFO DAGScheduler: Got job 12 (take at DeltaSync.java:441) with 1 output partitions 20/10/07 20:27:07 INFO DAGScheduler: Final stage: ResultStage 36 (take at DeltaSync.java:441) 20/10/07 20:27:07 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 35) 20/10/07 20:27:07 INFO DAGScheduler: Missing parents: List() 20/10/07 20:27:07 INFO DAGScheduler: Submitting ResultStage 36 (MapPartitionsRDD[65] at filter at DeltaSync.java:441), which has no missing parents 20/10/07 20:27:07 INFO MemoryStore: Block broadcast_18 stored as values in memory (estimated size 328.0 KB, free 1607.1 MB) 20/10/07 20:27:07 INFO MemoryStore: Block broadcast_18_piece0 stored as bytes in memory (estimated size 102.9 KB, free 1607.0 MB) 20/10/07 20:27:07 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 102.9 KB, free: 1608.4 MB) 20/10/07 20:27:07 INFO SparkContext: Created broadcast 18 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:07 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 36 (MapPartitionsRDD[65] at filter at DeltaSync.java:441) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:27:07 INFO YarnClusterScheduler: Adding task set 36.0 with 1 tasks 20/10/07 20:27:07 INFO TaskSetManager: Starting task 0.0 in stage 36.0 (TID 91, ip-172-31-19-77.ap-southeast-2.compute.internal, executor 16, partition 0, PROCESS_LOCAL, 7651 bytes) 20/10/07 20:27:07 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on ip-172-31-19-77.ap-southeast-2.compute.internal:39787 (size: 102.9 KB, free: 5.0 GB) 20/10/07 20:27:07 INFO TaskSetManager: Finished task 0.0 in stage 36.0 (TID 91) in 74 ms on ip-172-31-19-77.ap-southeast-2.compute.internal (executor 16) (1/1) 20/10/07 20:27:07 INFO YarnClusterScheduler: Removed TaskSet 36.0, whose tasks have all completed, from pool 20/10/07 20:27:07 INFO DAGScheduler: ResultStage 36 (take at DeltaSync.java:441) finished in 0.113 s 20/10/07 20:27:07 INFO DAGScheduler: Job 12 finished: take at DeltaSync.java:441, took 0.115583 s 20/10/07 20:27:07 ERROR DeltaSync: Global error : 20/10/07 20:27:08 INFO SparkContext: Starting job: collect at ListingBasedRollbackHelper.java:77 20/10/07 20:27:08 INFO DAGScheduler: Registering RDD 67 (mapToPair at ListingBasedRollbackHelper.java:103) as input to shuffle 8 20/10/07 20:27:08 INFO DAGScheduler: Got job 13 (collect at ListingBasedRollbackHelper.java:77) with 1 output partitions 20/10/07 20:27:08 INFO DAGScheduler: Final stage: ResultStage 38 (collect at ListingBasedRollbackHelper.java:77) 20/10/07 20:27:08 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 37) 20/10/07 20:27:08 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 37) 20/10/07 20:27:08 INFO DAGScheduler: Submitting ShuffleMapStage 37 (MapPartitionsRDD[67] at mapToPair at ListingBasedRollbackHelper.java:103), which has no missing parents 20/10/07 20:27:08 INFO MemoryStore: Block broadcast_19 stored as values in memory (estimated size 140.8 KB, free 1606.8 MB) 20/10/07 20:27:08 INFO MemoryStore: Block broadcast_19_piece0 stored as bytes in memory (estimated size 45.0 KB, free 1606.8 MB) 20/10/07 20:27:08 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 45.0 KB, free: 1608.4 MB) 20/10/07 20:27:08 INFO SparkContext: Created broadcast 19 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:08 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 37 (MapPartitionsRDD[67] at mapToPair at ListingBasedRollbackHelper.java:103) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:27:08 INFO YarnClusterScheduler: Adding task set 37.0 with 1 tasks 20/10/07 20:27:08 INFO TaskSetManager: Starting task 0.0 in stage 37.0 (TID 92, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 10, partition 0, PROCESS_LOCAL, 7776 bytes) 20/10/07 20:27:08 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:37747 (size: 45.0 KB, free: 5.0 GB) 20/10/07 20:27:09 INFO TaskSetManager: Finished task 0.0 in stage 37.0 (TID 92) in 1236 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 10) (1/1) 20/10/07 20:27:09 INFO YarnClusterScheduler: Removed TaskSet 37.0, whose tasks have all completed, from pool 20/10/07 20:27:09 INFO DAGScheduler: ShuffleMapStage 37 (mapToPair at ListingBasedRollbackHelper.java:103) finished in 1.263 s 20/10/07 20:27:09 INFO DAGScheduler: looking for newly runnable stages 20/10/07 20:27:09 INFO DAGScheduler: running: Set() 20/10/07 20:27:09 INFO DAGScheduler: waiting: Set(ResultStage 38) 20/10/07 20:27:09 INFO DAGScheduler: failed: Set() 20/10/07 20:27:09 INFO DAGScheduler: Submitting ResultStage 38 (MapPartitionsRDD[69] at map at ListingBasedRollbackHelper.java:77), which has no missing parents 20/10/07 20:27:09 INFO MemoryStore: Block broadcast_20 stored as values in memory (estimated size 5.6 KB, free 1606.8 MB) 20/10/07 20:27:09 INFO MemoryStore: Block broadcast_20_piece0 stored as bytes in memory (estimated size 3.1 KB, free 1606.8 MB) 20/10/07 20:27:09 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 3.1 KB, free: 1608.4 MB) 20/10/07 20:27:09 INFO SparkContext: Created broadcast 20 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:09 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 38 (MapPartitionsRDD[69] at map at ListingBasedRollbackHelper.java:77) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:27:09 INFO YarnClusterScheduler: Adding task set 38.0 with 1 tasks 20/10/07 20:27:09 INFO TaskSetManager: Starting task 0.0 in stage 38.0 (TID 93, ip-172-31-30-101.ap-southeast-2.compute.internal, executor 2, partition 0, NODE_LOCAL, 7651 bytes) 20/10/07 20:27:09 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on ip-172-31-30-101.ap-southeast-2.compute.internal:42829 (size: 3.1 KB, free: 5.0 GB) 20/10/07 20:27:10 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 8 to 172.31.30.101:59130 20/10/07 20:27:10 INFO TaskSetManager: Finished task 0.0 in stage 38.0 (TID 93) in 1037 ms on ip-172-31-30-101.ap-southeast-2.compute.internal (executor 2) (1/1) 20/10/07 20:27:10 INFO YarnClusterScheduler: Removed TaskSet 38.0, whose tasks have all completed, from pool 20/10/07 20:27:10 INFO DAGScheduler: ResultStage 38 (collect at ListingBasedRollbackHelper.java:77) finished in 1.045 s 20/10/07 20:27:10 INFO DAGScheduler: Job 13 finished: collect at ListingBasedRollbackHelper.java:77, took 2.310805 s 20/10/07 20:27:11 INFO SparkContext: Starting job: foreach at HoodieSparkEngineContext.java:79 20/10/07 20:27:11 INFO DAGScheduler: Got job 14 (foreach at HoodieSparkEngineContext.java:79) with 1 output partitions 20/10/07 20:27:11 INFO DAGScheduler: Final stage: ResultStage 39 (foreach at HoodieSparkEngineContext.java:79) 20/10/07 20:27:11 INFO DAGScheduler: Parents of final stage: List() 20/10/07 20:27:11 INFO DAGScheduler: Missing parents: List() 20/10/07 20:27:11 INFO DAGScheduler: Submitting ResultStage 39 (ParallelCollectionRDD[70] at parallelize at HoodieSparkEngineContext.java:79), which has no missing parents 20/10/07 20:27:11 INFO MemoryStore: Block broadcast_21 stored as values in memory (estimated size 124.9 KB, free 1606.6 MB) 20/10/07 20:27:11 INFO MemoryStore: Block broadcast_21_piece0 stored as bytes in memory (estimated size 36.3 KB, free 1606.6 MB) 20/10/07 20:27:11 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on ip-172-31-20-51.ap-southeast-2.compute.internal:35667 (size: 36.3 KB, free: 1608.4 MB) 20/10/07 20:27:11 INFO SparkContext: Created broadcast 21 from broadcast at DAGScheduler.scala:1203 20/10/07 20:27:11 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 39 (ParallelCollectionRDD[70] at parallelize at HoodieSparkEngineContext.java:79) (first 15 tasks are for partitions Vector(0)) 20/10/07 20:27:11 INFO YarnClusterScheduler: Adding task set 39.0 with 1 tasks 20/10/07 20:27:11 INFO TaskSetManager: Starting task 0.0 in stage 39.0 (TID 94, ip-172-31-17-203.ap-southeast-2.compute.internal, executor 6, partition 0, PROCESS_LOCAL, 7871 bytes) 20/10/07 20:27:11 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on ip-172-31-17-203.ap-southeast-2.compute.internal:44687 (size: 36.3 KB, free: 5.0 GB) 20/10/07 20:27:12 INFO TaskSetManager: Finished task 0.0 in stage 39.0 (TID 94) in 980 ms on ip-172-31-17-203.ap-southeast-2.compute.internal (executor 6) (1/1) 20/10/07 20:27:12 INFO YarnClusterScheduler: Removed TaskSet 39.0, whose tasks have all completed, from pool 20/10/07 20:27:12 INFO DAGScheduler: ResultStage 39 (foreach at HoodieSparkEngineContext.java:79) finished in 1.000 s 20/10/07 20:27:12 INFO DAGScheduler: Job 14 finished: foreach at HoodieSparkEngineContext.java:79, took 1.001778 s 20/10/07 20:27:12 ERROR HoodieDeltaStreamer: Got error running delta sync once. Shutting down org.apache.hudi.exception.HoodieException: Commit 20201007202651 failed and rolled-back ! at org.apache.hudi.utilities.deltastreamer.DeltaSync.writeToSink(DeltaSync.java:449) at org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:249) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$2(HoodieDeltaStreamer.java:163) at org.apache.hudi.common.util.Option.ifPresent(Option.java:96) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:161) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:466) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:685) 20/10/07 20:27:12 INFO Javalin: Stopping Javalin ... 20/10/07 20:27:12 INFO Javalin: Javalin has stopped 20/10/07 20:27:12 INFO SparkUI: Stopped Spark web UI at http://ip-172-31-20-51.ap-southeast-2.compute.internal:34477 20/10/07 20:27:12 INFO YarnClusterSchedulerBackend: Shutting down all executors 20/10/07 20:27:12 INFO YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down 20/10/07 20:27:12 INFO SchedulerExtensionServices: Stopping SchedulerExtensionServices (serviceOption=None, services=List(), started=false) 20/10/07 20:27:12 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped! 20/10/07 20:27:12 INFO MemoryStore: MemoryStore cleared 20/10/07 20:27:12 INFO BlockManager: BlockManager stopped 20/10/07 20:27:12 INFO BlockManagerMaster: BlockManagerMaster stopped 20/10/07 20:27:12 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped! 20/10/07 20:27:12 INFO SparkContext: Successfully stopped SparkContext 20/10/07 20:27:12 ERROR ApplicationMaster: User class threw exception: org.apache.hudi.exception.HoodieException: Commit 20201007202651 failed and rolled-back ! org.apache.hudi.exception.HoodieException: Commit 20201007202651 failed and rolled-back ! at org.apache.hudi.utilities.deltastreamer.DeltaSync.writeToSink(DeltaSync.java:449) at org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:249) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$2(HoodieDeltaStreamer.java:163) at org.apache.hudi.common.util.Option.ifPresent(Option.java:96) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:161) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:466) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:685) 20/10/07 20:27:12 INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: org.apache.hudi.exception.HoodieException: Commit 20201007202651 failed and rolled-back ! at org.apache.hudi.utilities.deltastreamer.DeltaSync.writeToSink(DeltaSync.java:449) at org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:249) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$2(HoodieDeltaStreamer.java:163) at org.apache.hudi.common.util.Option.ifPresent(Option.java:96) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:161) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:466) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:685) ) 20/10/07 20:27:12 INFO ApplicationMaster: Unregistering ApplicationMaster with FAILED (diag message: User class threw exception: org.apache.hudi.exception.HoodieException: Commit 20201007202651 failed and rolled-back ! at org.apache.hudi.utilities.deltastreamer.DeltaSync.writeToSink(DeltaSync.java:449) at org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:249) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$2(HoodieDeltaStreamer.java:163) at org.apache.hudi.common.util.Option.ifPresent(Option.java:96) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:161) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:466) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:685) ) 20/10/07 20:27:12 INFO AMRMClientImpl: Waiting for application to be successfully unregistered. 20/10/07 20:27:12 INFO ApplicationMaster: Deleting staging directory hdfs://ip-xxx-xx-xx-xxx.ap-xxx-x.compute.internal:8020/user/hadoop/.sparkStaging/application_1601158208025_9843 20/10/07 20:27:12 INFO ShutdownHookManager: Shutdown hook called 20/10/07 20:27:12 INFO ShutdownHookManager: Deleting directory /mnt/yarn/usercache/hadoop/appcache/application_1601158208025_9843/spark-8a15f98e-cc02-41af-8f3d-5100fb8ded79 20/10/07 20:27:12 INFO ShutdownHookManager: Deleting directory /mnt2/yarn/usercache/hadoop/appcache/application_1601158208025_9843/spark-559704ab-996d-4693-b9de-109221baf0ba 20/10/07 20:27:12 INFO ShutdownHookManager: Deleting directory /mnt1/yarn/usercache/hadoop/appcache/application_1601158208025_9843/spark-b32df694-aae4-49e0-a108-6c1b42d0b85c 20/10/07 20:27:12 INFO ShutdownHookManager: Deleting directory /mnt3/yarn/usercache/hadoop/appcache/application_1601158208025_9843/spark-67789bac-d9c6-43b7-a74a-30d1754c9bdd