Skip to content
Compare
Choose a tag to compare

Magpie 2.2

@chu11 chu11 released this
· 100 commits to master since this release
Compare
Choose a tag to compare
  • Support SPARK_IO_COMPRESSION_CODEC in Spark.
  • Add new submission script type "sbatchmpirun"
  • With sbatch & srun, add experimental support for distributed tensorflow support.
  • With sbatch & mpirun, add experimental support for Tensorflow with Horovod
  • Support new MAGPIE_PRE_EXECUTION_RUN and MAGPIE_POST_EXECUTION_RUN options. These options are similar to MAGPIE_PRE_JOB_RUN and MAGPIE_POST_JOB_RUN, but run before/after the entire software stack has been brought up / torn down.
  • Support Spark 2.4.1-bin-hadoop2.6, 2.4.1-bin-hadoop2.7
  • Update Spark default to 2.4.1-bin-hadoop2.7