Add a package for Picard

[hartzell]

See the discussion about installing jar files in #4386.

Also installs a wrapper script that has explicit references to the prerequisite java exe and to the jar file in it's final resting place.

[samfux84]

@hartzell What are the benefits of hard-coding the path to the java executable into the wrapper scripts ? If you provide multiple java versions their corresponding modules should set the $JAVA_HOME variable.

For Python and Perl scripts, I am using shebang lines, which do not hard-code the interpreter executable, but rather take the first version that they find in the environment (in module files paths are usually prepended instead of appended, therefore you should get the desired version of the interpreter, when you load a module):

#!/usr/bin/env python
#!/usr/bin/env perl

This way you do not restrict the script to a single version of the interpreter. For java a similar setup would be to use

$JAVA_HOME/bin/java

instead of using the full path.

[adamjstewart]

What are the benefits of hard-coding the path to the java executable into the wrapper scripts?

One of Spack's goals is that you should always run an executable with the dependencies it was built with. For shared objects, this means adding dependent libraries with RPATH. For scripting languages like Python, this means hard-coding the path to the python executable in the shebang. Running with versions other than the one you built it with is not supported and can lead to a lot of nasty headaches.

[hartzell]

@samfux84, thanks for asking!

What are the benefits of hard-coding the path...

Put briefly, because I want to be sure that the applications that I deploy reliably work.

There's no guarantee that some random perl binary (and the core that it brings with it) is going to be compatible with the various modules that make up my installed application.

At one job that favored the #!/usr/bin/env perl approach it happened so often that I had canned responses to it.

The most dramatic failures involve modules that include compiled C code, trying to run something compiled with perl@5.x.y crashes and burns when you run it with perl@5.x+n.y (for n != 0). perl's compiled with -fPIC or not, etc.... perl's compiled on CentOS 7 using libraries compiled on CentOS 6. Etc...

Even pure perl modules aren't really safe, if you install an application with a perl that includes a particular package (perhaps at a particular version) in its core then try to run it with a different perl where that package was removed from the core (or perhaps its an incompatible version), kerblooie. And worse than kerblooie is the time spent headscratching and blame-gaming and ....

Proper perl packaging (thank goodness I have a pop filter on my keyboard) has you starting your scripts with #!perl and then the installation machinery (e.g. cpanm) takes care of re-writing that for you so that it points to the perl that was used to do the installation.

The two exceptions to this are fatpacked applications (e.g. ack) and scripts that are so so so simple that "nothing could possibly go wrong".

Of course, creative individuals can still wreak havoc via PERL5LIB, but there's only so much that I can do. As an aside, I've played with using sitecustomize.pl scripts (search the perlrun docs for sitecustomize) that would, in certain situations, disable PERL5LIB, but I could never get my cow-orkers onboard.

The other dynamic languages have similar issues.

I'm less experienced with Java (and I don't do Windows...), but I see no reason to let users monkey up a perfectly good application because they have something funny in their .bashrc (:cold_sweat:).

[samfux84]

@hartzell @adamjstewart Thank you both for your answers.

I am still in the process of switching from manual installations to using spack. In the previous software stack that I have setup manually (about 260 software packages, 180 Python modules, 120 R packages) I tried to keep everything as flexible as possible, avoiding to hard-code things wherever possible.

For me switching to RPATH and hard-coding things wherever possible is a paradigm change that needs some time to get used to.

It makes sense to restrict the number of ways a cluster user can do things wrong to a minimum.

[hartzell]

keep everything as flexible as possible

One of the beautiful things about this kind of infrastructure/automation is:

• that it let's us hang on to the flexibility we need (it's simple to reconfigure and redeploy); and also
• provides the reproducibility and reliability (explicit wiring of dependencies, audit-ablility, etc...) that we crave.