Spark CLASSPATH

◀️ KryoSerializer Generator 🔗 Home ▶️

---

The Spark class path is defined in the following order:

$CLASSPATH
$PADOGRID_WORKSPACE/clusters/<cluster>/plugins/*
$PADOGRID_WORKSPACE/clusters/<cluster>/lib/*
$PADOGRID_WORKSPACE/plugins/*
$PADOGRID_WORKSPACE/lib/*
$PADOGRID_HOME/spark/plugins/*
$PADOGRID_HOME/spark/lib/*
$PADOGRID_HOME/lib/*
$SPARK_HOME/jars/*

To see worker startup details, run the start_worker -simulate command, which does not run the worker but simulates the startup sequence that includes displaying the order of CLASSPATH, Java options, and arguments.

start_worker -simulate

Note that CLASSPATH and JAVA_OPTS can be set in setenv.sh at each directory level to include additional paths or override any paths set in the subsequent directory levels. The setenv.sh files are located as follows:

$PADOGRID_WORKSPACE/clusters/$CLUSTER/bin_sh/setenv.sh
$PADOGRID_WORKSPACE/setenv.sh
$PADOGRID_HOME/bin_sh/setenv.sh

---

◀️ KryoSerializer Generator 🔗 Home ▶️