Skip to content

Commit

Permalink
don't require jupyter for installation (#791)
Browse files Browse the repository at this point in the history
* work on jupyter-free installation

* rebase fix

* drop compat, save jupyter pref again, add env to installkernel

* updated notebook command, also jupyterlab command adapted from #783

* installkernel test

* clean up test output

* whoops

* rm another compat reference

* rm more compatibility code

* updated docs

* installkernel is exported

* add .exe suffix on Windows

* fix path for windows

* no compat

* only prompt once for jupyterlab install
  • Loading branch information
stevengj committed Jan 3, 2019
1 parent 2211a9c commit f813325
Show file tree
Hide file tree
Showing 15 changed files with 371 additions and 401 deletions.
86 changes: 49 additions & 37 deletions README.md
Expand Up @@ -11,7 +11,9 @@ environment (also used by [IPython](http://ipython.org/)). This
combination allows you to interact with the Julia language using
Jupyter/IPython's powerful [graphical
notebook](http://ipython.org/notebook.html), which combines code,
formatted text, math, and multimedia in a single document.
formatted text, math, and multimedia in a single document. It
also works with [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/), a Jupyter-based
integrated development environment for notebooks and code.

(IJulia notebooks can also be re-used in other Julia code via
the [NBInclude](https://github.com/stevengj/NBInclude.jl) package.)
Expand All @@ -28,38 +30,16 @@ Pkg.add("IJulia")
```
to install IJulia.

By default on Mac and Windows, the `Pkg.add` process will use the [Conda.jl](https://github.com/Luthaf/Conda.jl)
package to install a minimal Python+Jupyter distribution (via
[Miniconda](http://conda.pydata.org/docs/install/quick.html)) that is
private to Julia (not in your `PATH`). (You can use `using IJulia` followed by
`IJulia.jupyter` to find the location `jupyter` where was installed.)
On Linux, it defaults to looking for `jupyter` in your `PATH` first,
and only installs the Conda Jupyter if that fails; you can force
it to use Conda on Linux by setting `ENV["JUPYTER"]=""` first (see below).
(In a Debian or Ubuntu GNU/Linux system, install the package `jupyter-client`
first to use the system `jupyter`.)

Alternatively, you can [install
Jupyter](http://jupyter.readthedocs.org/en/latest/install.html) (or
IPython 3 or later) yourself *before* adding the IJulia package.
To tell IJulia to use your *own* `jupyter` installation, you need
to set `ENV["JUPYTER"]` to the path of the `jupyter` program
before running `Pkg.add("IJulia")` (if jupyter is in your PATH, you can also just use `ENV["JUPYTER"]="jupyter"`). Alternatively, you can change
which `jupyter` program IJulia is configured with by setting
`ENV["JUPYTER"]` and then running `Pkg.build("IJulia")`.

The simplest way to install Jupyter yourself on Mac and Windows, other
than using Julia's Conda distro, is to [download
the Anaconda package](http://continuum.io/downloads) and run its
installer. (We recommend that you *not* use Enthought Canopy/EPD
since that can cause problems with the PyCall package.)

On subsequent builds (e.g. when IJulia is updated via `Pkg.update`),
it will use the same `jupyter` program by default, unless you
override it by setting the `JUPYTER` environment variable, or
delete the file `joinpath(Pkg.dir("IJulia"), "deps", "JUPYTER")`.
You can go back to using the Conda `jupyter` by setting
`ENV["JUPYTER"]=""` and re-running `Pkg.build("IJulia")`.
This process installs a [kernel specification](https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernelspecs) that tells Jupyter (or JupyterLab) etcetera
how to launch Julia.

`Pkg.add("IJulia")` does not actually install Jupyter itself.
You can install Jupyter if you want, but it can also be installed
automatically when you run `IJulia.notebook()` below. (You
can force it to use a specific `jupyter` installation by
setting `ENV["JUPYTER"]` to the path of the `jupyter` program
before `Pkg.add`, or before running `Pkg.build("IJulia")`;
your preference is remembered on subsequent updates.

### Running the IJulia Notebook

Expand All @@ -68,7 +48,19 @@ In Julia, at the `julia>` prompt, you can type
using IJulia
notebook()
```
to launch the IJulia notebook in your browser. You can
to launch the IJulia notebook in your browser.

The first time you run `notebook()`, it will prompt you
for whether it should install Jupyter. Hit enter to
have it use the [Conda.jl](https://github.com/Luthaf/Conda.jl)
package to install a minimal Python+Jupyter distribution (via
[Miniconda](http://conda.pydata.org/docs/install/quick.html)) that is
private to Julia (not in your `PATH`).
On Linux, it defaults to looking for `jupyter` in your `PATH` first,
and only asks to installs the Conda Jupyter if that fails; you can force
it to use Conda on Linux by setting `ENV["JUPYTER"]=""` during installation (see above). (In a Debian or Ubuntu GNU/Linux system, install the package `jupyter-client` to install the system `jupyter`.)

You can
use `notebook(detached=true)` to launch a notebook server
in the background that will persist even when you quit Julia.
This is also useful if you want to keep using the current Julia
Expand Down Expand Up @@ -103,13 +95,25 @@ on the *New* button and choose the *Julia* option to start a new
"notebook". A notebook will combine code, computed results, formatted
text, and images, just as in IPython. You can enter multiline input
cells and execute them with *shift-ENTER*, and the menu items are
mostly self-explanatory. Refer to [the IPython
documentation](http://ipython.org/documentation.html) for more
mostly self-explanatory. Refer to [the Jupyter notebook
documentation](https://jupyter-notebook.readthedocs.io/en/stable/) for more
information, and see also the "Help" menu in the notebook itself.

Given an IJulia notebook file, you can execute its code within any
other Julia file (including another notebook) via the [NBInclude](https://github.com/stevengj/NBInclude.jl) package.

### Running the JupyterLab

Instead of running the classic notebook interface, you can use the IDE-like JupyterLab. This can be launched from within Julia via:

```jl
using IJulia
jupyterlab()
```

Like `notebook()`, this will install JupyterLab via Conda if it is
not installed already. `jupyterlab()` also supports `detached` and `dir` keyword options similar to `notebook()`.

### Updating Julia and IJulia

Julia is improving rapidly, so it won't be long before you want to
Expand Down Expand Up @@ -143,12 +147,20 @@ For example, if you want to run Julia with all deprecation warnings
disabled, you can do:
```julia
using IJulia
IJulia.installkernel("Julia nodeps", "--depwarn=no")
installkernel("Julia nodeps", "--depwarn=no")
```
and a kernel called `Julia nodeps 0.7` (if you are using Julia 0.7)
will be installed (will show up in your main Jupyter kernel menu) that
lets you open notebooks with this flag.

You can also install kernels to run Julia with different environment
variables, for example to set [`JULIA_NUM_THREADS`](https://docs.julialang.org/en/v1/manual/environment-variables/index.html#JULIA_NUM_THREADS-1) for use with Julia [multithreading](https://docs.julialang.org/en/v1/manual/parallel-computing/#Multi-Threading-(Experimental)-1):
```
using IJulia
installkernel("Julia (4 threads)", env=Dict("JULIA_NUM_THREADS"=>"4"))
```
The `env` keyword should be a `Dict` mapping environment variables to values.

### Troubleshooting:

* If you ran into a problem with the above steps, after fixing the
Expand Down
1 change: 0 additions & 1 deletion REQUIRE
Expand Up @@ -2,6 +2,5 @@ julia 0.7
MbedTLS 0.4.3
JSON 0.17
ZMQ 1.0.0
Compat 0.69.0
Conda 0.1.5
SoftGlobalScope
142 changes: 28 additions & 114 deletions deps/build.jl
@@ -1,129 +1,43 @@
#######################################################################
import JSON, Conda
using Compat
using Compat.Unicode: lowercase
using Conda

jupyter=""

# remove deps.jl at exit if it exists, in case build.jl fails
try
#######################################################################
# Install Jupyter kernel-spec file.
include("kspec.jl")
kernelpath = installkernel("Julia")

# Make sure Python uses UTF-8 output for Unicode paths
ENV["PYTHONIOENCODING"] = "UTF-8"
# make it easier to get more debugging output by setting JULIA_DEBUG=1
# when building.
IJULIA_DEBUG = lowercase(get(ENV, "IJULIA_DEBUG", "0"))
IJULIA_DEBUG = IJULIA_DEBUG in ("1", "true", "yes")

function prog_version(prog)
v = try
chomp(read(`$prog --version`, String))
catch
return nothing
end
try
return VersionNumber(v)
catch
Compat.@warn("`$jupyter --version` returned an unrecognized version number $v")
return v"0.0"
end
end

# remember the user's Jupyter preference, if any; empty == Conda
prefsfile = joinpath(first(DEPOT_PATH), "prefs", "IJulia")
mkpath(dirname(prefsfile))

global jupyter = get(ENV, "JUPYTER", isfile(prefsfile) ? readchomp(prefsfile) : Compat.Sys.isunix() && !Compat.Sys.isapple() ? "jupyter" : "")
if isempty(jupyter)
jupyter_vers = nothing
else
jupyter_vers = prog_version(jupyter)
if jupyter_vers === nothing
jupyter_vers = prog_version(jupyter * "-notebook")
end
if jupyter_vers === nothing
Compat.@warn("Could not execute `$jupyter --version`.")
jupyter = get(ENV, "JUPYTER", isfile(prefsfile) ? readchomp(prefsfile) : Sys.isunix() && !Sys.isapple() ? "jupyter" : "")
condajupyter = normpath(Conda.SCRIPTDIR, exe("jupyter"))
if !isempty(jupyter)
if dirname(jupyter) == abspath(Conda.SCRIPTDIR)
jupyter = condajupyter # will be installed if needed
end
end
isconda = dirname(jupyter) == abspath(Conda.SCRIPTDIR)
if Sys.ARCH in (:i686, :x86_64) && (jupyter_vers === nothing || jupyter_vers < v"3.0" || isconda)
isconda || jupyter_vers === nothing || Compat.@info("$jupyter was too old: got $jupyter_vers, required ≥ 3.0")
Compat.@info("Installing Jupyter via the Conda package.")
Conda.add("jupyter")
jupyter = abspath(Conda.SCRIPTDIR, "jupyter")
jupyter_vers = prog_version(jupyter)
end
if jupyter_vers === nothing || jupyter_vers < v"3.0"
error("Failed to find or install Jupyter 3.0 or later. Please install Jupyter manually, set `ENV[\"JUPYTER\"]=\"/path/to/jupyter\", and rerun `Pkg.build(\"IJulia\")`.")
end
Compat.@info("Found Jupyter version $jupyter_vers: $jupyter")

#######################################################################
# Get the latest syntax highlighter file.
if isconda
highlighter = joinpath(Conda.LIBDIR, "python2.7", "site-packages", "notebook", "static",
"components", "codemirror", "mode", "julia", "julia.js")
# CodeMirror commit from which we get the syntax highlighter
cm_commit = "ed9278cba6e1f75328df6b257f1043d35a690c59"
highlighter_url = "https://raw.githubusercontent.com/codemirror/CodeMirror/" *
cm_commit * "/mode/julia/julia.js"
if isfile(highlighter)
try
download(highlighter_url, highlighter)
catch e
Compat.@warn("The following error occurred while attempting to download latest ",
"syntax highlighting definitions:\n\n", e, "\n\nSyntax highlighting may ",
"not work as expected.")
if isabspath(jupyter)
if !Sys.isexecutable(jupyter)
@warn("ignoring non-executable JUPYTER=$jupyter")
jupyter = condajupyter
end
elseif Sys.which(jupyter) === nothing
@warn("JUPYTER=$jupyter not found in PATH")
end
end

#######################################################################
# Warn people upgrading from older IJulia versions:
try
juliaprof = chomp(read(pipeline(`$ipython locate profile julia`,
stderr=devnull), String))
Compat.@warn("""You should now run IJulia just via `$jupyter notebook`, without
the `--profile julia` flag. IJulia no longer maintains the profile.
Consider deleting $juliaprof""")
catch
function write_if_changed(filename, contents)
if !isfile(filename) || read(filename, String) != contents
write(filename, contents)
end
end

#######################################################################
# Install Jupyter kernel-spec file.

include("kspec.jl")
kspec_cmd, = installkernel("Julia")

# figure out the notebook command by replacing (only!) the last occurrence of
# "kernelspec" with "notebook":
notebook = kspec_cmd.exec
n = notebook[end]
ki = findlast("kernelspec", n)
notebook[end] = n[1:prevind(n,first(ki))] * "notebook" * n[nextind(n,last(ki)):end]

#######################################################################
# make it easier to get more debugging output by setting JULIA_DEBUG=1
# when building.
IJULIA_DEBUG = lowercase(get(ENV, "IJULIA_DEBUG", "0"))
IJULIA_DEBUG = IJULIA_DEBUG in ("1", "true", "yes")

#######################################################################
# Install the deps.jl file:

if v"4.2" jupyter_vers < v"5.1"
# disable broken data-rate limit (issue #528)
push!(notebook, "--NotebookApp.iopub_data_rate_limit=2147483647")
end
deps = """
const jupyter = "$(escape_string(jupyter))"
const notebook_cmd = ["$(join(map(escape_string, notebook), "\", \""))"]
const jupyter_vers = $(repr(jupyter_vers))
const IJULIA_DEBUG = $(IJULIA_DEBUG)
"""
if !isfile("deps.jl") || read("deps.jl", String) != deps
write("deps.jl", deps)
end
write(prefsfile, jupyter)

#######################################################################
catch
isfile("deps.jl") && rm("deps.jl") # remove deps.jl file on build error
rethrow()
end
const JUPYTER = $(repr(jupyter))
"""
write_if_changed("deps.jl", deps)
write_if_changed(prefsfile, jupyter)

0 comments on commit f813325

Please sign in to comment.