Error: syntax: undefined reference in AST

[MasonProtter]

Suddenly when I try to start up IJulia and run any code in a Jupyter notebook or jupyter lab (e.g. just return the number 1), I get an error:

syntax: undefined reference in AST

Stacktrace:
 [1] top-level scope at In[1]:0
 [2] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091
 [3] execute_code(::String, ::String) at /home/mason/.julia/packages/IJulia/a1SNk/src/execute_request.jl:27
 [4] execute_request(::ZMQ.Socket, ::IJulia.Msg) at /home/mason/.julia/packages/IJulia/a1SNk/src/execute_request.jl:86
 [5] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [6] invokelatest at ./essentials.jl:709 [inlined]
 [7] eventloop(::ZMQ.Socket) at /home/mason/.julia/packages/IJulia/a1SNk/src/eventloop.jl:8
 [8] (::IJulia.var"#15#18")() at ./task.jl:356

I am on a rolling release Linux distro (manjaro) so I suspect that a update to Jupyter or one of Jupyter's dependencies broke this.

Does anyone have advice to help me diagnose this issue? I'd really appreciate the help. I tried updating IJulia updating julia, running build IJulia, deleting and reinstalling IJulia, etc.

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1* (2020-11-09 13:37 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: AMD Ryzen 5 2600 Six-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, znver1)
Environment:
  JULIA_NUM_THREADS = 6

(@v1.5) pkg> st IJulia
Status `~/.julia/environments/v1.5/Project.toml`
  [7073ff75] IJulia v1.22.0

[Babber]

Same problem here. I updated from v1.5.2 which was working fine. v1.5.3 also works fine in REPL, but in JupyterLab, it is completely broken as reported above.

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

[Babber]

This solved it for me:

1. Close all Julia sessions
2. Remove all content of ~/.julia/packages/IJulia/ and ~/.julia/compiled/v1.5/IJulia/
3. In Julia REPL (can be the latest v1.5.3):

using Pkg
Pkg.add(Pkg.PackageSpec(;name="IJulia", version="1.21.3"))

4. Launch Jupyter, it should work now.

[MasonProtter]

Yep, that solved it, thanks @Babber!

[Babber]

It's great to hear that it helped you, @MasonProtter. This workaround moves back IJulia by two releases, there is however still a bug or incompatibility in the more recent versions, so I suggest to leave this issue open. Do you agree?

[stevengj]

I can't reproduce on my Mac.

It sounds like the error is happening at the Julia end, not the Jupyter end, but to verify this you could try with the Anaconda Jupyter instead. (ENV["JUPYTER"]="" and re-build IJulia, quit Julia, run using IJulia; notebook(), and say yes when it asks you if you want to download and install Jupyter.)

[stevengj]

(What does status print at the pkg> prompt? Have you pinned any other packages at old versions?)

[MasonProtter]

I tried updating and following those instructions Steven and the error persisted. I did this in an empty environment (and without a startup_ijulia file):

(temp) pkg> st
Status `~/temp/Project.toml`
  [7073ff75] IJulia v1.22.0

but in case you need it, here is my full environment (I do not believe I have any relevant packages out of date):

(@v1.5) pkg> st
Status `~/.julia/environments/v1.5/Project.toml`
  [537997a7] AbstractPlotting v0.12.18
  [79e6a3ab] Adapt v2.3.0
  [7d9fca2a] Arpack v0.4.0
  [6e4b80f9] BenchmarkTools v0.5.0
  [e2ed5e7c] Bijections v0.1.2
  [ad839575] Blink v0.12.3
  [b546bd83] BoseHubbard v0.1.0 `~/Dropbox/Research/GeometricPhase/BoseHubbard`
  [052768ef] CUDA v2.1.0
  [7057c7e9] Cassette v0.3.3
  [082447d4] ChainRules v0.7.32
  [9706b775] ChainedFixes v0.2.1
  [da1fd8a2] CodeTracking v1.0.5
  [34da2185] Compat v3.23.0
  [459fdd68] ComputedFieldTypes v0.1.0
  [cd731e1f] Conjugates v0.1.0
  [85a47980] Dictionaries v0.3.2
  [0c46a032] DifferentialEquations v6.15.0
  [31c24e10] Distributions v0.24.3
  [e2ba6199] ExprTools v0.1.3
  [cc61a311] FLoops v0.1.4
  [1a297f60] FillArrays v0.9.7
  [26cc04aa] FiniteDifferences v0.11.2
  [6cd66ae4] FoldsCUDA v0.1.1
  [f6369f11] ForwardDiff v0.10.12
  [0c68f7d7] GPUArrays v6.1.1
  [28b8d3ca] GR v0.53.0
  [14197337] GenericLinearAlgebra v0.2.4
  [19dc6840] HCubature v1.4.0
  [7073ff75] IJulia v1.22.0
  [7869d1d1] IRTools v0.4.1
  [95701278] ImplicitEquations v1.0.2
  [c601a237] Interact v0.10.3
  [41ab1584] InvertedIndices v1.0.0
  [c8e1da08] IterTools v1.3.0
  [033835bb] JLD2 v0.2.4
  [0f8b85d8] JSON3 v1.4.0
  [5cadff95] JuliennedArrays v0.2.2
  [63c18a36] KernelAbstractions v0.4.5
  [b964fa9f] LaTeXStrings v1.2.0
  [5078a376] LazyArrays v0.18.1
  [bdcacae8] LoopVectorization v0.8.26
  [d8e11817] MLStyle v0.4.6
  [1914dd2f] MacroTools v0.5.6
  [ee78f7c6] Makie v0.11.1
  [1154507a] Mjolnir v0.2.0 `~/.julia/dev/Mjolnir`
  [961ee093] ModelingToolkit v3.21.0
  [872cf16e] NaturallyUnitful v0.3.0
  [429524aa] Optim v1.2.0
  [bac558e1] OrderedCollections v1.3.2
  [8314cec4] PGFPlotsX v1.2.10
  [9b87118b] PackageCompiler v1.2.3
  [d96e819e] Parameters v0.12.1
  [8afb3109] PatternDispatch v0.1.0 `https://github.com/MasonProtter/PatternDispatch.jl#master`
  [58dd65bb] Plotly v0.3.0
  [f0f68f2c] PlotlyJS v0.14.0
  [91a5bcdd] Plots v1.8.1
  [c3e4b0f8] Pluto v0.11.14 `~/.julia/dev/Pluto`
  [438e738f] PyCall v1.92.1
  [d330b81b] PyPlot v2.9.0
  [74087812] Random123 v1.2.0
  [b873ce64] ReplMaker v0.2.4 `~/.julia/dev/ReplMaker`
  [295af30f] Revise v3.1.7
  [88634af6] SaferIntegers v2.5.1
  [efcf1570] Setfield v0.7.0
  [23f74ba5] Spec v0.1.0 `../../../Dropbox/Julia/Spec`
  [90137ffa] StaticArrays v0.12.5
  [0e73dd9d] StaticModules v0.1.0 `~/.julia/dev/StaticModules`
  [d8176aec] StaticRanges v0.8.3
  [2913bbd2] StatsBase v0.33.2
  [09ab397b] StructArrays v0.4.4
  [8e1ec7a9] SumTypes v0.1.0 `../../../SumTypes`
  [123dc426] SymEngine v0.8.2
  [d1185830] SymbolicUtils v0.5.2
  [52b65841] TaggedArrays v0.1.0 `../../../TaggedArrays`
  [6aa5eb33] TaylorSeries v0.10.8
  [02d47bb6] TensorCast v0.3.2
  [6aa20fa7] TensorOperations v1.3.1
  [ac1d9e8a] ThreadsX v0.1.7
  [28d57a85] Transducers v0.4.53
  [410a4b4d] Tricks v0.1.3
  [bc48ee85] Tullio v0.2.9
  [7b1b5591] TullioBLAS v0.1.0 `../../../Dropbox/Julia/TullioBLAS`
  [d9a01c3f] Underscores v2.0.0
  [1cfade01] UnicodeFun v0.4.1
  [b8865327] UnicodePlots v1.3.0
  [1986cc42] Unitful v1.5.0
  [f7972512] VarStructs v0.4.0 `https://github.com/aminya/VarStructs.jl#master`
  [0f1e0344] WebIO v0.8.15
  [c2297ded] ZMQ v1.2.1
  [e88e6eb3] Zygote v0.5.9
  [37e2e46d] LinearAlgebra
  [de0858da] Printf
  [9a3f8284] Random
  [2f01184e] SparseArrays

[stevengj]

The only other thing I could suggest would be to put some @show statements in the body of the include_string function to see what it is doing, i.e. add @show ast, @show xf, etcetera.

[Paalon]

I have same symptoms in my macOS.

Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

Working on [7073ff75] IJulia v1.21.3, broken on [7073ff75] IJulia v1.22.0.

Status `~/.julia/environments/v1.5/Project.toml`
  [6e4b80f9] BenchmarkTools v0.5.0
  [7e6ae17a] BioSequences v2.0.5
  [336ed68f] CSV v0.7.7
  [052768ef] CUDA v2.1.0
  [a93c6f00] DataFrames v0.21.8
  [31c24e10] Distributions v0.23.12
  [587475ba] Flux v0.11.2
  [7073ff75] IJulia v1.21.3
  [c8e1da08] IterTools v1.3.0
  [033835bb] JLD2 v0.2.4
  [b964fa9f] LaTeXStrings v1.2.0
  [1dea7af3] OrdinaryDiffEq v5.42.3
  [91a5bcdd] Plots v1.8.1
  [92933f4c] ProgressMeter v1.4.0
  [438e738f] PyCall v1.92.1
  [93e0c654] Reduce v1.2.9
  [295af30f] Revise v3.1.7
  [276daf66] SpecialFunctions v0.10.3
  [24249f21] SymPy v1.0.32
  [bd369af6] Tables v1.2.1
  [1d978283] TensorFlow v0.11.0
  [b8865327] UnicodePlots v1.3.0
  [cb13b1c6] Waveforms v0.1.1
  [e88e6eb3] Zygote v0.5.9

[stevengj]

Can you try doing

using Conda
Conda.update()

and then relaunch Jupyter in case it is something stemming from an out-of-date Conda installation?

[Paalon]

I tried updating Conda and relaunching Jupyter but the same error exists in IJulia v1.22.0.

# println("hello") in Jupyter Notebook, then

syntax: undefined reference in AST

Stacktrace:
 [1] top-level scope at In[1]:0
 [2] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091
 [3] execute_code(::String, ::String) at /Users/paalon/.julia/packages/IJulia/a1SNk/src/execute_request.jl:27
 [4] execute_request(::ZMQ.Socket, ::IJulia.Msg) at /Users/paalon/.julia/packages/IJulia/a1SNk/src/execute_request.jl:86
 [5] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [6] invokelatest at ./essentials.jl:709 [inlined]
 [7] eventloop(::ZMQ.Socket) at /Users/paalon/.julia/packages/IJulia/a1SNk/src/eventloop.jl:8
 [8] (::IJulia.var"#15#18")() at ./task.jl:356

[stevengj]

Since I can't reproduce, my only suggestion is to dig into the IJulia source code a bit and print some debugging info as I said above.

[Paalon]

Change code to

        # use the default REPL ast transformations in Julia 1.5,
        # which include the soft-scope transformation.
        include_string(current_module[], code, "In[$n]") do ast
            for xf in REPL.repl_ast_transforms
                @show ast
                @show xf
                if xf !== REPL.softscope || SOFTSCOPE[]
                    ast = Base.invokelatest(xf, ast)
                end
            end
            ast
        end

then

In [1]: 1

gives

ast = :(#= In[1]:1 =#)
xf = Revise.revise_first

syntax: undefined reference in AST

Stacktrace:
 [1] top-level scope at In[1]:0
 [2] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091
 [3] execute_code(::String, ::String) at /Users/paalon/git/IJulia.jl/src/execute_request.jl:27
 [4] execute_request(::ZMQ.Socket, ::IJulia.Msg) at /Users/paalon/git/IJulia.jl/src/execute_request.jl:88
 [5] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [6] invokelatest at ./essentials.jl:709 [inlined]
 [7] eventloop(::ZMQ.Socket) at /Users/paalon/git/IJulia.jl/src/eventloop.jl:8
 [8] (::IJulia.var"#15#18")() at ./task.jl:356