Merge branch 'master' of github.com:JuliaLang/julia

DISTRIBUTING.md

@@ -10,7 +10,7 @@ order to create a portable, working Julia distribution, we have
 separated most of the notes by OS.
 
 Note that while the code for Julia is
-[MIT-licensed](https://github.com/JuliaLang/julia/blob/master/LICENSE.md),
+[MIT-licensed, with a few exceptions](https://github.com/JuliaLang/julia/blob/master/LICENSE.md),
 the distribution created by the techniques described herein will be
 GPL licensed, as various dependent libraries such as `FFTW`, `Rmath`,
 `SuiteSparse`, and `git` are GPL licensed. We do hope to have a
@@ -26,7 +26,7 @@ pregenerate the `base/version_git.jl` file with:
 
     make -C base version_git.jl.phony
 
-Juila has lots of build dependencies where we use patched versions that has not
+Julia has lots of build dependencies where we use patched versions that has not
 yet been included by the popular package managers. These dependencies will usually
 be automatically downloaded when you build, but if you want to be able to build
 Julia on a computer without internet access you should create a source-dist archive
@@ -43,6 +43,25 @@ to create a Make.user file containing:
 
     override TAGGED_RELEASE_BANNER = "my-package-repository build"
 
+Target Architectures
+--------------------
+
+By default, Julia optimizes its system image to the native architecture of
+the build machine. This is usually not what you want when building packages,
+as it will make Julia fail at startup on any machine with incompatible CPUs
+(in particular older ones with more restricted instruction sets).
+
+We therefore recommend that you pass the `MARCH` variable when calling `make`,
+setting it to the baseline target you intend to support. This will determine
+the target CPU for both the Julia executable and libraries, and the system
+image (the latter can also be set using `JULIA_CPU_TARGET`). Typically useful
+values for x86 CPUs are `x86-64` and `core2` (for 64-bit builds) and
+`pentium4` (for 32-bit builds). Unfortunately, CPUs older than Pentium 4
+are currently not supported (see
+[this issue](https://github.com/JuliaLang/julia/issues/7185)).
+
+The full list of CPU targets supported by LLVM can be obtained by running
+`llc -mattr=help`.
 
 Linux
 -----
@@ -52,10 +71,17 @@ installation. If you wish to create a distribution package such as a
 `.deb`, or `.rpm`, some extra effort is needed. See the
 [julia-debian](http://github.com/staticfloat/julia-debian) repository
 for an example of what metadata is needed for creating `.deb` packages
-for Debian and Ubuntu-based systems. Although we have not yet experimented
+for Debian and Ubuntu-based systems. See the
+[Fedora package](https://admin.fedoraproject.org/pkgdb/package/julia/)
+for RPM-based distributions. Although we have not yet experimented
 with it, [Alien](https://wiki.debian.org/Alien) could be used to
 generate Julia packages for various Linux distributions.
 
+Julia supports overriding standard installation directories via `prefix`
+and other environment variables you can pass when calling `make` and
+`make install`. See Make.inc for their list. `DESTDIR` can also be used
+to force the installation into a temporary directory.
+
 By default, Julia loads `$prefix/etc/julia/juliarc.jl` as an
 installation-wide initialization file. This file can be used by
 distribution managers to provide paths to various binaries such as a
@@ -102,8 +128,8 @@ Notes on BLAS and LAPACK
 Julia builds OpenBLAS by default, which includes the BLAS and LAPACK
 libraries. On 32-bit architectures, Julia builds OpenBLAS to use
 32-bit integers, while on 64-bit architectuers, Julia builds OpenBLAS
-to use 64-bit integers. It is essential that all Julia functions that
-call BLAS and LAPACK API routines use integers of the correct width.
+to use 64-bit integers (ILP64). It is essential that all Julia functions
+that call BLAS and LAPACK API routines use integers of the correct width.
 
 Most BLAS and LAPACK distributions provided on linux distributions,
 and even commercial implementations ship libraries that use 32-bit
@@ -120,6 +146,22 @@ all libraries that depend on BLAS and LAPACK. The Julia build process
 will build all these libraries correctly, but when overriding defaults
 and using system provided libraries, this consistency must be ensured.
 
+Also note that Linux distributions sometimes ship several versions of
+OpenBLAS, some of which enable multithreading, and others only working
+in a serial fashion. For example, in Fedora, `libopenblasp.so` is threaded,
+but `libopenblas.so` is not. We recommend using the former for optimal
+performance. To choose an OpenBLAS library whose name is different from
+the default `libopenblas.so`, pass `LIBBLAS=-l$(YOURBLAS)` and
+`LIBBLASNAME=lib$(YOURBLAS)` to `make`, replacing `$(YOURBLAS)` with the
+name of your library. You can also add `.so.0` to the name of the library
+if you want your package to work without requiring the unversioned `.so`
+symlink.
+
+Finally, OpenBLAS includes its own optimized version of LAPACK. If you
+set `USE_SYSTEM_BLAS=1` and `USE_SYSTEM_LAPACK=1`, you should also set
+`LIBLAPACK=-l$(YOURBLAS)` and `LIBLAPACKNAME=lib$(YOURBLAS)`. Else, the
+reference LAPACK will be used and performance will typically be much lower.
+
 Notes on Rmath
 ==============
 
@@ -136,7 +178,8 @@ used.
 Compilation scripts
 ===================
 
-The [julia-nightly-packaging](https://github.com/staticfloat/julia-nightly-packaging) repository contains multiple example scripts to ease the creation of
+The [julia-nightly-packaging](https://github.com/staticfloat/julia-nightly-packaging)
+repository contains multiple example scripts to ease the creation of
 binary packages. It also includes miscellaneous tools to do things such as
 fetching the last good commit that passed the
 [Travis](https://travis-ci.org/JuliaLang/julia/builds) tests.

README.md

@@ -76,7 +76,7 @@ Once it is built, you can run the `julia` executable using its full path in the
 
 - add a soft link to the `julia` executable in the `julia` directory to `/usr/local/bin` (or any suitable directory already in your path), or
 
-- add the `julia` directory to your executable path for this shell session (in bash: `export PATH="$(pwd):$PATH"` ; in csh or tcsh:
+- add the `julia` directory to your executable path for this shell session (in `bash`: `export PATH="$(pwd):$PATH"` ; in `csh` or `tcsh`:
 `set path= ( $path $cwd )` ), or
 
 - add the `julia` directory to your executable path permanently (e.g. in `.bash_profile`), or
@@ -91,7 +91,7 @@ If everything works correctly, you will see a Julia banner and an interactive pr
 
 Your first test of Julia should be to determine whether your
 build is working properly. From the UNIX/Windows command prompt inside
-the julia source directory, type `make testall`. You should see output
+the `julia` source directory, type `make testall`. You should see output
 that lists a series of tests being run; if they complete without
 error, you should be in good shape to start using Julia.
 
@@ -101,6 +101,52 @@ If you are building a Julia package for distribution on Linux, OS X,
 or Windows, take a look at the detailed notes in
 [DISTRIBUTING.md](https://github.com/JuliaLang/julia/blob/master/DISTRIBUTING.md).
 
+### Updating an existing source tree
+
+If you have previously downloaded `julia` using `git clone`, you can update the
+existing source tree using `git pull` rather than starting anew:
+
+    cd julia
+    git pull && make
+
+Assuming that you had made no changes to the source tree that will conflict
+with upstream updates, these commands will trigger a build to update to the
+latest version.
+
+#### General troubleshooting
+
+1. Over time, the base library may accumulate enough changes such that the
+   bootstrapping process in building the system image will fail. If this
+   happens, the build may fail with an error like
+
+   ```sh
+    *** This error is usually fixed by running 'make clean'. If the error persists, try 'make cleanall' ***
+   ```
+
+   As described, running `make clean && make` is usually sufficient.
+   Occasionally, the stronger cleanup done by `make cleanall` is needed.
+
+2. New versions of external dependencies may be introduced which may
+   occasionally cause conflicts with existing builds of older versions.
+
+   a. Special `make` targets exist to help wipe the existing build of a
+      dependency. For example, `make -C deps clean-llvm` will clean out the
+      existing build of `llvm`.
+
+   b. To delete existing binaries of `julia` and all its dependencies,
+      delete the `./usr` directory _in the source tree_.
+
+3. In extreme cases, you may wish to reset the source tree to a pristine state.
+   The following git commands may be helpful:
+
+   ```sh
+    git reset --hard #Forcibly remove any changes to any files under version control
+    git clean -x -f -d #Forcibly remove any file or directory not under version control
+   ```
+
+   _To avoid losing work, make sure you know what these commands do before you
+   run them. `git` will not be able to undo these changes!_
+
 <a name="Uninstalling-Julia"/>
 ## Uninstalling Julia
 
@@ -122,7 +168,7 @@ Julia does not install anything outside the directory it was cloned into. Julia
 
 Julia can be built for a non-generic architecture by configuring the `ARCH` Makefile variable. See the appropriate section of `Make.inc` for additional customization options, such as `MARCH` and `JULIA_CPU_TARGET`.
 
-For example, to build for i486, set `ARCH=i486` and install the necessary system libraries for linking. On Ubuntu, these may include lib32gfortran3 (also manually call `ln -s /usr/lib32/libgfortran3.so.0 /usr/lib32/libgfortran3.so`) and lib32gcc1, lib32stdc++6, among others.
+For example, to build for Pentium 4, set `MARCH=pentium4` and install the necessary system libraries for linking. On Ubuntu, these may include lib32gfortran3 (also manually call `ln -s /usr/lib32/libgfortran3.so.0 /usr/lib32/libgfortran3.so`) and lib32gcc1, lib32stdc++6, among others.
 
 You can also set `MARCH=native` for a maximum-performance build customized for the current machine CPU.
 
@@ -265,20 +311,23 @@ SuiteSparse is a special case, since it is typically only installed as a static
 
 ### Intel compilers and Math Kernel Libraries
 
-To use the Intel [MKL] BLAS and LAPACK libraries, make sure that MKL version 10.3.6 or higher is installed. For a 64-bit architecture, the MKL environment should be set up as follows:
+To use the Intel [MKL] BLAS and LAPACK libraries, make sure that MKL version 10.3.6 or higher is installed.
+For a 64-bit architecture, the environment should be set up as follows:
 
+    # bash
     source /path/to/mkl/bin/mklvars.sh intel64 ilp64
     export MKL_INTERFACE_LAYER=ILP64
 
-Julia can be built with the Intel compilers and MKL using the following flags.
+It is recommended that Intel compilers be used to build julia when using MKL.
+Add the following to the `Make.user` file:
 
     USEICC = 1
     USEIFC = 1
     USE_INTEL_MKL = 1
     USE_INTEL_MKL_FFT = 1
     USE_INTEL_LIBM = 1
 
-It is highly recommended to use a fresh clone of the Julia repository. Also, it is recommended that Intel compilers be used to build julia when using MKL.
+It is highly recommended to start with a fresh clone of the Julia repository.
 
 <a name="Source-Code-Organization"/>
 ## Source Code Organization