Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Plots.jl on Clusters #519

Closed
ChrisRackauckas opened this issue Oct 5, 2016 · 20 comments
Closed

Plots.jl on Clusters #519

ChrisRackauckas opened this issue Oct 5, 2016 · 20 comments
Labels
wontfix

Comments

@ChrisRackauckas
Copy link
Member

ChrisRackauckas commented Oct 5, 2016 •
edited

Does anyone have a way to get Plots.jl backends working on standard clusters? Any of them? I am talking about SGE/slurm clusters (likely accessed via SSH, X11 forwarding sometimes enabled, like XSEDE Comet, Bridges, etc.) where some standard components like Python, matplotlib, GTK, etc. are contained in (non-Julia) modules which you have to import. I've done the "native" thing of module avail for any module which makes sense, and running the Julia code, but Plots.jl can't find a backend. As of right now, I don't know of a way to even get plots to save. Here's the lowdown:

  • I wouldn't expect Plotly or PlotlyJS to work on these kind of clusters since there's no graphical browser available. But could they be used to save a plot even when there's no browser? There is some "Elinks" thing that shows up when Plotly plots (PlotlyJS isn't able to plot because Electron won't build), could that be used to save a plot?
  • PyPlot can find matplotlib by using the Conda.jl install method, but it can't find a backend, even after things like GTK and Python have been made available. Does the matplotlibrc file need to be modified in this case @stevengj?
  • @jheinen GR also can't load all of the libraries:
julia> plot(rand(4,4))
ERROR: error compiling _plot!: error compiling _display: could not load library "/home/crackauc/.julia/v0.5/GR/deps/gr/lib/libGR.so"
/lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/crackauc/.julia/v0.5/GR/deps/gr/lib/libGR.so)
 in plot(::Array{Float64,2}) at /home/crackauc/.julia/v0.5/Plots/src/plot.jl:46
  • PGFPlots can't find a backend either:
julia> pgfplots()
INFO: Precompiling module PGFPlots.
Plots.PGFPlotsBackend()
julia> plot(rand(4,4))
WARNING: _show is not defined for this backend. m=text/plain
  • UnicodePlots isn't publication quality (sorry @Evizero), but also, the terminals don't usually support unicode and give some odd output.

Anyone have a working method for Plots.jl on a cluster like this?
@ChrisRackauckas
Copy link
Member Author

If you're in the US and want a testing allocation, you can look into getting one from XSEDE:

https://portal.xsede.org/allocations/announcements#trial

The main computers are Stampede and Comet (with the new Bridges being cool too). These would be some good systems to test on (pretty standard setups).

@jheinen
Copy link
Member

jheinen commented Oct 5, 2016 •
edited

I managed to get Plots.jl and GR working on JURECA (http://www.fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JURECA/JURECA_node.html), a supercomputer with 1872 compute nodes based on a CentOS 7 Linux distribution.

These were the necessary steps (I don't have access to the system directories, so i tried an installation in my home directory):

  • Download and unpack Julia 0.5
  • Initialize the package repository and add the Plots and GR packages:
    julia> Pkg.init()
    julia> Pkg.add("Plots)
    julia> Pkg.add("GR")
  • Checkout GR master: Pkg.checkout("GR")
  • Force a download of the pre-built run-time:
    julia> ENV["GRDIR"]=""
    julia> Pkg.build("GR")
    INFO: Building GR
    INFO: Downloading pre-compiled GR latest Linux binary
  • Restart Julia and perform a simple test:
    julia> using Plots; gr(); histogram(randn(10000))

screen shot 2016-10-05 at 11 06 16

The above mentioned Linux binary (provided by GR) should work on all RPM based Linux systems. The build script also provides dedicated versions for Ubuntu and Debian. However, problems might occur, if the OS is too old.

@ChrisRackauckas : What is the software archiutecture of the SGE/slurm clusters you are using?

@ChrisRackauckas
Copy link
Member Author

It can depend on the cluster. But I think most of them are SGE/slurm on CentOS. I usually use the XSEDE Bridges, Stampede, Comet, or my local UCI cluster (also CentOS). I'll give this a try and report back.

@Evizero
Copy link
Member

Evizero commented Oct 5, 2016

FWIW unicodeplots has two ascii based canvas

@ChrisRackauckas
Copy link
Member Author

ChrisRackauckas commented Oct 5, 2016 •
edited

@Evizero How do you use the ASCII canvas with Plots?

@stevengj
Copy link
Contributor

stevengj commented Oct 5, 2016

For PyPlot, in a headless setup you could use the Agg backend. (e.g. set ENV["MPLBACKEND"]="Agg")

@ChrisRackauckas
Copy link
Member Author

ChrisRackauckas commented Oct 6, 2016 •
edited

I can't seem to get the headless PyPlot to work @stevengj. This is right after doing the suggested install:

julia> using Plots

julia> ENV["MPLBACKEND"]="Agg"
"Agg"

julia> pyplot()
INFO: Recompiling stale cache file /home/crackauc/.julia/lib/v0.5/PyCall.ji for module PyCall.
INFO: Recompiling stale cache file /home/crackauc/.julia/lib/v0.5/PyPlot.ji for module PyPlot.
WARNING: Couldn't initialize pyplot.  (might need to install it?)
INFO: To do a standard install of pyplot, copy and run this:

if !Plots.is_installed("PyPlot")
    Pkg.add("PyPlot")
end
withenv("PYTHON" => "") do
    Pkg.build("PyPlot")
end


ERROR: InitError: PyError (:PyImport_ImportModule) <type 'exceptions.ImportError'>
ImportError("/lib64/libz.so.1: version `ZLIB_1.2.3.4' not found (required by /home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/../../../libpng16.so.16)",)
  File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/colorbar.py", line 34, in <module>
    import matplotlib.collections as collections
  File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/collections.py", line 27, in <module>
    import matplotlib.backend_bases as backend_bases
  File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 62, in <module>
    import matplotlib.textpath as textpath
  File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/textpath.py", line 18, in <module>
    from matplotlib.mathtext import MathTextParser
  File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/mathtext.py", line 60, in <module>
    import matplotlib._png as _png

 in pyerr_check at /home/crackauc/.julia/v0.5/PyCall/src/exception.jl:56 [inlined]
 in pyerr_check at /home/crackauc/.julia/v0.5/PyCall/src/exception.jl:61 [inlined]
 in macro expansion at /home/crackauc/.julia/v0.5/PyCall/src/exception.jl:81 [inlined]
 in pyimport(::String) at /home/crackauc/.julia/v0.5/PyCall/src/PyCall.jl:387
 in __init__() at /home/crackauc/.julia/v0.5/PyPlot/src/PyPlot.jl:235
 in _include_from_serialized(::String) at ./loading.jl:150
 in _require_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:187
 in require(::Symbol) at ./loading.jl:394
 in _initialize_backend(::Plots.PyPlotBackend) at /home/crackauc/.julia/v0.5/Plots/src/backends/pyplot.jl:63
 in backend() at /home/crackauc/.julia/v0.5/Plots/src/backends.jl:151
 in #pyplot#137(::Array{Any,1}, ::Function) at /home/crackauc/.julia/v0.5/Plots/src/backends.jl:20
 in pyplot() at /home/crackauc/.julia/v0.5/Plots/src/backends.jl:20
during initialization of module PyPlot

@ChrisRackauckas
Copy link
Member Author

@jheinen

julia> ENV["GRDIR"]=""
""

julia> Pkg.build("GR")
INFO: Building GR

It looks like that isn't downloading the pre-compiled binary for XSEDE Comet. Is there a way to force it?

@jheinen
Copy link
Member

jheinen commented Oct 6, 2016 •
edited

@ChrisRackauckas

Hmm, setting GRDIR to an empty strong is the way to force it.

See file GR/deps/build.jl:

import Compat
if "GRDIR" in keys(ENV)
    have_dir = length(ENV["GRDIR"]) > 0
elseif ...
end
if !have_dir
  ...
  tarball = "gr-$version-$os-$arch.tar.gz"
  if !isfile("downloads/$tarball")
    info("Downloading pre-compiled GR $version $os binary")
    mkpath("downloads")
    download("http://gr-framework.org/downloads/$tarball", "downloads/$tarball")
    if os == :Windows
       ...
    else
      run(`tar xzf downloads/$tarball`)
      rm("downloads/$tarball")
    end
  end
end

Is there already an older tar file in the downloads folder? On some clusters, the download may hang due to firewall restrictions. Can you download the file manually?

julia> download("http://gr-framework.org/downloads/gr-latest-Linux-x86_64.tar.gz")

@ChrisRackauckas
Copy link
Member Author

Yeah, that's command causes a download. There was no tar in the GR/deps/download folder though.

@jheinen
Copy link
Member

jheinen commented Oct 6, 2016

@ChrisRackauckas : Could you - if you have time - debug the deps/build.j script? On our clusters it works and I have no access to your machines. Thanks in advance ...

In case you just want to prove GR, simply unpack the tar file in the GR deps directory and you can probably test Plots :-)

@ChrisRackauckas
Copy link
Member Author

I unpacked the library but got:

ERROR: error compiling #plot_data#4: error compiling clearws: could not load library "/home/crackauc/.julia/v0.5/GR/deps/gr/lib/libGR.so"
/lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/crackauc/.julia/v0.5/GR/deps/gr/lib/libGR.so)
 in (::GR.jlgr.#kw##plot_data)(::Array{Any,1}, ::GR.jlgr.#plot_data) at ./<missing>:0
 in #plot#20(::Array{Any,1}, ::Function, ::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/jlgr.jl:848
 in plot(::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/jlgr.jl:840
 in #plot#6(::Array{Any,1}, ::Function, ::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/GR.jl:1244
 in plot(::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/GR.jl:1244

when calling GR directly.

@jheinen
Copy link
Member

jheinen commented Oct 9, 2016

@ChrisRackauckas
What is the output of ldd deps/gr/lib/libGR.so?

@ChrisRackauckas
Copy link
Member Author 

[crackauc@comet-ln3 GR]$ ldd deps/gr/lib/libGR.so
deps/gr/lib/libGR.so: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by deps/gr/lib/libGR.so)
        linux-vdso.so.1 =>  (0x00007ffc031f2000)
        libXt.so.6 => /usr/lib64/libXt.so.6 (0x00007f8303711000)
        libX11.so.6 => /usr/lib64/libX11.so.6 (0x00007f83033d4000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f83031b7000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f8302fb2000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f8302c1e000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f830299a000)
        libSM.so.6 => /usr/lib64/libSM.so.6 (0x00007f8302791000)
        libICE.so.6 => /usr/lib64/libICE.so.6 (0x00007f8302575000)
        libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x00007f8302357000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003cef600000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f8302152000)
        libXau.so.6 => /usr/lib64/libXau.so.6 (0x00007f8301f4f000)

@ChrisRackauckas
Copy link
Member Author

For reference, gcc is available and the available modules are:

[crackauc@comet-ln3 GR]$ module avail

-------------------------------------------------------------------------------- /opt/modulefiles/applications/.intel --------------------------------------------------------------------------------
atlas/3.10.2(default)     gsl/1.16                  ipm/2.0.3(default)        netcdf/4.3.2(default)     pdt/3.20(default)         sprng/2.0b(default)       trilinos/11.12.1(default)
boost/1.55.0(default)     gsl/2.1(default)          lapack/3.6.0(default)     p3dfft/2.7.4(default)     petsc/3.6.3(default)      sundials/2.6.2(default)
fftw/2.1.5                hdf4/2.11(default)        mxml/2.9(default)         papi/5.4.1(default)       scalapack/2.0.2(default)  superlu/4.2(default)
fftw/3.3.4(default)       hdf5/1.8.14(default)      netcdf/3.6.2              parmetis/4.0.3(default)   slepc/3.6.2(default)      tau/2.23(default)

------------------------------------------------------------------------------------ /opt/modulefiles/mpi/.intel -------------------------------------------------------------------------------------
intelmpi/2016.3.210(default) mvapich2_gdr/2.1(default)    mvapich2_ib/2.1(default)     openmpi_ib/1.8.4(default)

------------------------------------------------------------------------------------- /opt/modulefiles/compilers -------------------------------------------------------------------------------------
cilk/5.4.6(default)           gnu/4.9.2(default)            intel/2013_sp1.2.144(default) intel/2016.3.210              pgi/15.7(default)
cmake/3.3.2(default)          guile/2.0.11(default)         intel/2015.2.164              mono/3.12.0(default)          python/2.7.10(default)

----------------------------------------------------------------------------------- /opt/modulefiles/applications ------------------------------------------------------------------------------------
abaqus/6.11-2                 bowtie2/2.2.7(default)        gamess/2014.12(default)       lammps/20160216(default)      node/0.12.6(default)          singularity/2.1.2(default)
abaqus/6.14-1(default)        bwa/0.7.13(default)           garli/2.01(default)           llvm/3.6.2(default)           nwchem/6.6(default)           soapdenovo/240(default)
abinit/7.10.5(default)        bx-python/0.7.3(default)      gateway-usage-reporting/1.0   mafft/7.187(default)          octave/4.0.0(default)         SOAPsnp/1.03(default)
abyss/1.5.2(default)          celera/8.3rc2(default)        gaussian/09.D.01(default)     matlab/2016a(default)         openbabel/2.3.2(default)      spades/3.9.0(default)
amber/15(default)             cp2k/3.0(default)             gaussian/09.E.01              matt/1.00(default)            picard/2.3.0(default)         spark/1.2.0
apbs/1.4.2(default)           cpmd/3.17.1(default)          gdal/1.11.2(default)          migrate/3.6.11(default)       plink/1.9(default)            spark/1.5.2(default)
bamtools/2.4.0(default)       cuda/6.5                      GenomeAnalysisTK/3.5          migrate/3.6.8                 polymake/2.14(default)        squid/1.9g(default)
bbcp/14.09.02.00.0(default)   cuda/7.0(default)             GenomeAnalysisTK/3.6(default) miRDeep2/0.0.7(default)       proj/4.9.1(default)           stacks/1.37(default)
bbftp/3.2.1(default)          cufflinks/2.2.1(default)      geos/3.4.2(default)           miso/0.5.3(default)           pysam/0.8.4(default)          tophat/2.1.1(default)
bcftools/1.3(default)         ddt/6.0.6(default)            globus/5.2.5                  mkl/11.1.2.144(default)       qchem/4.3.2(default)          trimmomatic/0.35(default)
beagle/2.1(default)           dendropy/4.0.3(default)       gmap_gsnap/20151231(default)  mkl/11.2.2.164                qe/5.3.0(default)             trinity/2.1.1(default)
beast/1.8.0                   diamond/0.7.12(default)       gmp/6.0.0a(default)           mkl/11.3.3                    qiime/1.9.1(default)          vasp/4.6
beast/1.8.1                   drive-data/6.2(default)       gnutools/2.69(default)        molden/5.0.7(default)         R/3.2.3(default)              vasp/5.4.1.20160205(default)
beast/1.8.2(default)          edena/3.131028(default)       grace/5.1.25(default)         mopac/2016(default)           randfold/2.0(default)         vcftools/0.1.14(default)
beast2/2.1.3(default)         eigen/3.2.7(default)          gromacs/5.1.2(default)        mpc/1.0.3(default)            rapidminer/7.1.0(default)     velvet/1.2.10(default)
bedtools/2.25.0(default)      eman2/2.1(default)            hadoop/1.2.1                  mpfr/3.1.2(default)           raxml/8.2.3(default)          ViennaRNA/2.2.4(default)
biopython/1.66(default)       emboss/6.5.7(default)         hadoop/2.6.0(default)         mpi4py/1.3.1(default)         relion/1.4(default)           visit/2.9.1(default)
biotools/4(default)           fastqc/0.11.4(default)        hmmer/3.1b2(default)          mpiblast/1.6.0(default)       rseqc/2.6.3(default)          vmd/1.9.2(default)
bismark/0.15.0(default)       fasttree/2.1.8(default)       htseq/0.6.1p1(default)        namd/2.10(default)            samtools/1.2                  weka/3.7.12(default)
blast/2.3.0(default)          fastx/0.0.14(default)         idba-ud/1.1.1(default)        namd/2.9                      samtools/1.3(default)
blat/35(default)              frealign/9.11(default)        idl/8.4(default)              ncl_ncarg/6.2.1(default)      scipy/2.7(default)
bowtie/1.1.2(default)         fsa/1.15.9(default)           jasper/1.900.1(default)       ncview/2.1.7(default)         siesta/3.2.6(default)

----------------------------------------------------------------------------------- /usr/share/Modules/modulefiles -----------------------------------------------------------------------------------
dot         module-git  module-info modules     null        use.own

@jheinen
Copy link
Member

jheinen commented Oct 9, 2016

@ChrisRackauckas : Hmm, so glibc is too old? What is the output of rpm -q -f /lib64/libc.so.6 and cat /etc/redhat-release?

@ChrisRackauckas
Copy link
Member Author 

[crackauc@comet-ln3 GR]$ rpm -q -f /lib64/libc.so.6
glibc-2.12-1.166.el6_7.7.x86_64
[crackauc@comet-ln3 GR]$ cat /etc/redhat-release
CentOS release 6.7 (Final)

@jheinen
Copy link
Member

jheinen commented Oct 9, 2016 •
edited

@ChrisRackauckas - Ok. I think we should better deploy the CentOS 6 build for the GR run-time. Right now, we use CentOS 7 for building the "pre-compiled GR latest Linux binary".

I will pre-check V6 compatibility with V7 (and other RPM-based Linux distributions) and tell you about the results when I'm back in office. The julia binary itself has similar dependency, so I expect no problems.

Thanks for testing!

@ChrisRackauckas
Copy link
Member Author

The GR changes plus #578 works really well. I am able to see the plots with UnicodePlots, re-make and save them using GR, and then send them back to my computer even without X11 forwarding. With X11 forwarding GR works fine. This makes a good workflow for plotting on common clusters. I've tried this now on 3 clusters with the same results: GR/Unicode works fine, Plotly(JS) of course doesn't because of the lack of web browsers, and PyPlot still has issues.

I think this should be added to the documentation on backends. What exactly do you think it should say. I want to say something like "Compatible with clusters based on CentOS 6/7 based SGE and Slurm clusters".

@stevengj
Copy link
Contributor

stevengj commented Mar 6, 2017

The PyPlot problem is due to multiple incompatible libz libraries: JuliaPy/PyPlot.jl#151

@ChrisRackauckas ChrisRackauckas mentioned this issue Feb 5, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tbreloff @ChrisRackauckas @stevengj @jheinen @Evizero