julia: Update to v1.1.1 with Portfile simplifications

[essandess]

julia: Update to v1.1.1 with Portfile simplifications

• Update to latest Julia release 1.1.1
• Simplify Portfile for easier maintenance
• Build all julia dependencies from source
• GPG verification
• Run make test

Description

The MacPorts julia port is years out of date.

This Portfile builds julia from source using all its own native dependencies, without the complications of using MacPorts library dependencies.

Type(s)

• bugfix
• enhancement
• security fix

Tested on

macOS 10.14
Xcode 8.x

Verification

Have you

• followed our Commit Message Guidelines?
• squashed and minimized your commits?
• checked that there aren't other open pull requests for the same change?
• referenced existing tickets on Trac with full URL?

• checked your Portfile with port lint?
• tried existing tests with sudo port test?
• tried a full install with sudo port -vst install?
• tested basic functionality of all binary files?

[macportsbot]

Travis Build #7301 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

Port julia's dependencies fail on xcode9.4. Log

The build timed out.

[pmetzger]

A few comments. Given the complexity of this, I'd like other people to review it if possible.

[essandess]

I'll need some help updating the compiler dependencies here.

Everything builds fine on my local box. On CI, I'm missing gfortran -> gfortran-mp-8, which comes from gcc8. I'm not sure what the compilers.choose fc f77 f90 and compilers.setup require_fortran -g95 are doing.

I'll try again with a port:gcc8 dependency.

Build fail from https://paste.z0k.xyz/d616f9d68886:

make[4]: gfortran: No such file or directory

[macportsbot]

Travis Build #7311 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

Port julia's dependencies fail on xcode9.4. Log

The build timed out.

[macportsbot]

Travis Build #7313 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

The build timed out.

[essandess]

This build is failing because there's no gfortran:

gfortran: No such file or directory

I believe that gfortran is created by port select --set gcc …, but @cjones051073 says that port select cannot be reliably used within a Portfile.

Is there an accepted, robust recipe for providing gfortran in a Portfile? Or just create my own symbolic link to gfortran-mp-9 and make sure it's in PATH?

[cjones051073]

This build is failing because there's no gfortran:

gfortran: No such file or directory

I believe that gfortran is created by port select --set gcc …, but @cjones051073 says that port select cannot be reliably used within a Portfile.

Is there an accepted, robust recipe for providing gfortran in a Portfile? Or just create my own symbolic link to gfortran-mp-9 and make sure it's in PATH?

Best option - Ask upstream if they provide a configuration option to set the fortran compiler. If so, just set this to ${prefix}/bin/gfortran-mp-9.0

[cjones051073]

I strongly disagree with the change "This Portfile builds julia from source using all its own native dependencies, without the complications of using MacPorts library dependencies.". As a point of principle we prefer ports to use the macports provided ports as dependencies, as far as possible. The fact you need gfortran here is infact because of the internal OpenBLAS build, which is not properly configured...

[essandess]

@cjones051073

I strongly disagree with the change "This Portfile builds julia from source using all its own native dependencies, without the complications of using MacPorts library dependencies.". As a point of principle we prefer ports to use the macports provided ports as dependencies

This approach/policy is a mistake in this instance, and preventing people like me from using MacPorts to install Julia. Why?

• MacPorts Julia is years out of date.
• MacPorts Julia uses libraries that may not be compatible or optimized for the latest Julia version, causing possible debugging or other issues.
• Good luck trying to get upstream support if you’re not using a build supported by the Julia team.

The Julia Project itself eschews MacPorts and other package managers for just this reason:

Although some system package managers provide Julia, such installations are neither maintained nor endorsed by the Julia project. They may be outdated, broken and/or unmaintained. We recommend you use the official Julia binaries instead.

This describes the Julia binaries provided by MacPorts ever since I can remember. The reason I’m updating this now is because it’s less trouble to manage my packages through a central manager like MacPorts, rather then separate installs for everything.

The solution to all these issues is simple: build Julia the way they say it should be built. Don’t complicate it with other stuff that may break the install and complicate updates.

If anyone wants to use their own libraries and figure all that out and manage it all, good on them.

But the default build should be just build it the way the package says it should be built without unnecessary complications.

[essandess]

@cjones051073 @pmetzger I need Portfile n00b help with another build issue, please.

The ${worksrcpath} is getting deleted between the build and destroot stages.

I do not understand why or how.

The build itself is simple: essentially make ; make install, both within ${worksrcpath}. I put the make step into the build phase, and the make install step into the destroot phase. But when the process gets to the make install In destroot, ${worksrcpath} has disappeared and the build fails.

Log excerpts:

--->  Building julia
DEBUG: Executing org.macports.build (julia)
DEBUG: Environment: 
CC_PRINT_OPTIONS='YES'
CC_PRINT_OPTIONS_FILE='/opt/local/var/macports/build/_opt_local_ports_lang_julia
/julia/work/.CC_PRINT_OPTIONS'
CPATH='/opt/local/include'
LIBRARY_PATH='/opt/local/lib'
MACOSX_DEPLOYMENT_TARGET='10.14'
Executing:  cd "/opt/local/var/macports/build/_opt_local_ports_lang_julia/julia/work/julia-1.1.1" && /usr/bin/make -j7 -w
…
--->  Staging julia into destroot
…
DEBUG: system -W /opt/local/var/macports/build/_opt_local_ports_lang_julia/julia/work/julia-1.1.1: /usr/bin/make -w install -j7 DESTDIR=/opt/local/var/macports/build/_opt_local_ports_lang_julia/julia/work/destroot
chdir: /opt/local/var/macports/build/_opt_local_ports_lang_julia/julia/work/julia-1.1.1: No such file or directory
Command failed: /usr/bin/make -w install -j7 DESTDIR=/opt/local/var/macports/build/_opt_local_ports_lang_julia/julia/work/destroot

[mf2k]

This build is failing because there's no gfortran:

gfortran: No such file or directory

I believe that gfortran is created by port select --set gcc …, but @cjones051073 says that port select cannot be reliably used within a Portfile.

And Chris is completely correct here. You need to tell the build system where to find gfortran as installed by Macports. A portfile should never depend on a port select setting.

[macportsbot]

Travis Build #7324 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

Port julia's dependencies fail on xcode9.4. Log
Port julia's dependencies fail on xcode7.3. Log

The build timed out.

[essandess]

The latest push builds successfully, and produces a binary for which all base tests works:

julia <<BASE_RUNTESTS_JL
Base.runtests()
BASE_RUNTESTS_JL

After a successful Buildbots build it's good to go.

[essandess]

I spoke too soon: one Base.runtests() fails, apparently because of Julia DESTDIR issues. I opened an issue upstream.

Nevertheless, I'd still argue to update this package to the latest version because aside from that issue everything I see works.

[macportsbot]

Travis Build #7333 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

Port julia's dependencies fail on xcode10.2. Log

The build timed out.

[macportsbot]

Travis Build #7336 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

Port julia's dependencies fail on xcode7.3. Log

The build timed out.

[essandess]

Updated Portfile with several improvements that address the compiler issue automatically and MacPorts settings in destroot.

[essandess]

@pmetzger @cjones051073 @mf2k

I discovered the .env options, which makes this build really easy and compliant with MacPorts's Portfile style.

I believe that this is really ready to commit now.

[macportsbot]

Travis Build #7365 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

Port julia's dependencies fail on xcode7.3. Log

The build timed out.

[macportsbot]

Travis Build #7373 Errored.

Lint results

--->  Verifying Portfile for julia
--->  0 errors and 0 warnings found.

Port julia's dependencies fail on xcode7.3. Log

The build timed out.

[essandess]

@mf2k @pmetzger @neverpanic @mojca @cjones051073

This port update works. I’ve had the Julia binaries deployed for nearly a month without issue. The previous version is years out of date.

May we please merge or identify any remaining issues?

[mf2k]

LGTM

@pmetzger: Are you ready to approve this?

[cjones051073]

I still remain not entirely convinced of the merits of the GPG stuff, but OK to let that go if others are happy.

[essandess]

@cjones051073 To be clear, gpg verification is not performed in the Portfile. I kept the code in an unused if-block to automate checking for maintainers if they activate the flag by hand. This is like a livecheck setting that isn’t used unless you ask for it.

I’m willing to go either way to get the update done. My own practice is to perform gpg signature verification if the package provides it, as Julia does.

[cjones051073]

@cjones051073 To be clear, gpg verification is not performed in the Portfile. I kept the code in an unused if-block to automate checking for maintainers if they activate the flag by hand. This is like a livecheck setting that isn’t used unless you ask for it.

I’m willing to go either way to get the update done. My own practice is to perform gpg signature verification if the package provides it, as Julia does.

Sorry, yes, I remember the discussion now (its been a while). I'm fine with this. If @pmetzger can OK the changes we are ready to go I would say.

[pmetzger]

I won't stand in the way (read: feel free to commit if everyone else is happy), but I'd like @essandess to move this support to a portgroup reasonably soon. It does not belong in an individual portfile; that makes it unpleasant to maintain.

[essandess]

I'd like @essandess to move this support to a portgroup reasonably soon.

I’d be happy to. Is there documentation somewhere how to write a portgroup?

Or a copy-and-paste-and-modify recommendation from an existing portgroup?

[cjones051073]

I’d be happy to. Is there documentation somewhere how to write a portgroup?

Or a copy-and-paste-and-modify recommendation from an existing portgroup?

https://guide.macports.org/chunked/reference.portgroup.html

[essandess]

@neverpanic @mf2k @pmetzger @mojca @cjones051073

Ping. May we please merge this?