Repository navigation
Initial working version
Pre-release
Pre-release
·
16 commits
to main
since this release
This code works with a couple of minor issues and runs in 1.2hrs for Mercator only when executed with the same resources as the production nightly build:
spark2-submit --class org.gbif.demo.SparkMapApp --master yarn --driver-memory 4G --conf spark.dynamicAllocation.enabled=false --executor-memory 48G --num-executors 100 --executor-cores 6 --conf spark.yarn.executor.memoryOverhead=4096 --conf spark.sql.shuffle.partitions=1200 --conf spark.network.timeout=600s spark-map-sql-1.0.0-2.3.0.jar

