C compiler is not installed

[tesujimath]

In order to install some packages, e.g. FFTW, a C compiler is required. There is not one in the Docker container. I am using version 0.6.0 of the container.

The container would be very much more useful if this sort of thing Just Worked (tm).

Here's the output to confirm:

julia> Pkg.add("FFTW")
INFO: Cloning cache of AbstractFFTs from https://github.com/JuliaMath/AbstractFFTs.jl.git
INFO: Updating cache of BinDeps...
INFO: Cloning cache of Compat from https://github.com/JuliaLang/Compat.jl.git
INFO: Cloning cache of FFTW from https://github.com/JuliaMath/FFTW.jl.git
INFO: Cloning cache of SHA from https://github.com/staticfloat/SHA.jl.git
INFO: Updating cache of URIParser...
INFO: Installing AbstractFFTs v0.1.0
INFO: Installing BinDeps v0.6.0
INFO: Installing Compat v0.26.0
INFO: Installing FFTW v0.0.2
INFO: Installing SHA v0.3.3
INFO: Installing URIParser v0.1.8
INFO: Building FFTW
INFO: Attempting to Create directory /home/guestsi/.julia/v0.6/FFTW/deps/downloads
INFO: Downloading file http://www.fftw.org/fftw-3.3.6-pl1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4081k  100 4081k    0     0  31812      0  0:02:11  0:02:11 --:--:--  8091
INFO: Done downloading file http://www.fftw.org/fftw-3.3.6-pl1.tar.gz
INFO: Attempting to Create directory /home/guestsi/.julia/v0.6/FFTW/deps/src
INFO: Attempting to Create directory /home/guestsi/.julia/v0.6/FFTW/deps
INFO: Directory /home/guestsi/.julia/v0.6/FFTW/deps already created
INFO: Attempting to Create directory /home/guestsi/.julia/v0.6/FFTW/deps/builds
INFO: Changing Directory to /home/guestsi/.julia/v0.6/FFTW/deps/builds
INFO: Attempting to Create directory /home/guestsi/.julia/v0.6/FFTW/deps/usr/lib
configure: error: in `/home/guestsi/.julia/v0.6/FFTW/deps/builds':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
==================================================[ ERROR: FFTW ]===================================================

LoadError: failed process: Process(`/home/guestsi/.julia/v0.6/FFTW/deps/src/fftw-3.3.6-pl1/configure --prefix=/home/guestsi/.julia/v0.6/FFTW/deps/builds --libdir=/home/guestsi/.julia/v0.6/FFTW/deps/usr/lib --bindir=/home/guestsi/.julia/v0.6/FFTW/deps/usr/bin --enable-shared --disable-fortran --disable-mpi --enable-threads --enable-sse2 --enable-fma`, ProcessExited(1)) [1]
while loading /home/guestsi/.julia/v0.6/FFTW/deps/build.jl, in expression starting on line 73

====================================================================================================================

==================================================[ BUILD ERRORS ]==================================================

WARNING: FFTW had build errors.

 - packages with build errors remain installed in /home/guestsi/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("FFTW")`
 - build a single package by running its `deps/build.jl` script

====================================================================================================================
INFO: Package database updated

[malmaud]

See #12 and #8. The problem is that someone can take this docker image and add the dependencies for the packages they need in a derivative image, but they can't easily remove tools they don't need.

The compiler toolchain in particular would increase the size of the image non-trivially.

[tesujimath]

That's a very good point. However, I do think there may be a good way to meet both desires, that is, small core image, and compiler-toolchain equipped image. Namely, have two separate images: julia-core, and julia-full (say).

From my point of view, for the organisation that I am consulting for, they can't really afford to be depending on an unofficial julia-full container, say if I made one, since after I move on, that wouldn't get maintained, and would fall behind the official julia-core.

So, would you consider offering both julia-core and julia-full as official images? The ability to install packages needing a compiler toolchain is clearly a general desire, so julia-full would add significant value.

[tesujimath]

@malmaud What did you think of my suggestion for a full image? It's just that we have to make a decision here about whether to continue to run Julia from a container. But without support for installing packages requiring a C compiler, it's not viable for us.

[malmaud]

Well, it's not up to me, ultimately :) But sure, I don't have anything against something like a julia-full image that has build-essential and other necessary libraries for popular packages, like hdf5-tools. If a PR was submitted to add that, I'd vote in favor of it.

[IanButterworth]

I just came across this issue specifically needing a c compiler and hdf5-tools during some arm CI testing using these images on drone.io, so I also think the julia-full image is a worthwhile idea.

I opened issue JuliaLang/julia#32259 before realizing the docker images weren't officially managed

[IanButterworth]

Is there any chance of a fix for this? I haven't managed to get sudo working on Drone for ARM CI testing, so can't get gcc installed.

[IanButterworth]

For now, gcc can be installed with:

kind: pipeline
name: linux-arm-1.3-RC2

platform:
 os: linux
 arch: arm

steps:
- name: build
  image: julia:1.3.0-rc2
  commands:
     - uname -a
     - apt-get update
     - apt-get install -y gcc g++
     - julia --project=. -e 'using Pkg; Pkg.build(); Pkg.test(coverage=true)'

[tianon]

It was a semi-conscious choice not to include gcc given that it adds a non-trivial amount of "bloat" to the image size for what I've understood to be a small percentage of Julia modules which require it installed.

What I'd recommend is a short, simple Dockerfile which builds on our image and adds gcc akin to what you've done in your YAML:

FROM julia:1.3
RUN apt-get update && apt-get install -y gcc g++ && rm -rf /var/lib/apt/lists/*

Combine that with Docker's automated builds service and it's reasonably easy to keep a child image of ours up-to-date with gcc pre-installed.

[PallHaraldsson]

In order to install some packages, e.g. FFTW, a C compiler is required.

FYI @tesujimath I'm not sure that's true any longer.

Maybe just close this issue?

More and more packages no longer need a C compiler (seemingly also (the new) PackageCompiler.jl, what the last issue here was about), e.g. FFTW, with JLLs used instead:

JuliaMath/FFTW.jl@82c835d

i.e. no longer using such lines: https://github.com/JuliaMath/FFTW.jl/blob/cc4b9f4cdd58042b49f895bf859c146a4373ae6b/deps/build.jl#L60

[tesujimath]

FYI @tesujimath I'm not sure that's true any longer.

Well, that's good news! Sounds like this issue is obsolete now. Yay!