Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to precompile CSV #895

Closed
mthelm85 opened this issue Feb 5, 2021 · 13 comments
Closed

Failed to precompile CSV #895

mthelm85 opened this issue Feb 5, 2021 · 13 comments

Comments

@mthelm85
Copy link
Contributor

mthelm85 commented Feb 5, 2021

using CSV in a Pluto notebook causes:

Failed to precompile CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b] to C:\Users\mthel\.julia\compiled\v1.5\CSV\HHBkp_tAWrN.ji.

error(::String)@error.jl:33
compilecache(::Base.PkgId, ::String)@loading.jl:1305
_require(::Base.PkgId)@loading.jl:1030
require(::Base.PkgId)@loading.jl:928
require(::Module, ::Symbol)@loading.jl:923
top-level scope@Local: 1

REPL output:

      From worker 2:    ERROR: LoadError: UndefVarError: invrefpool not defined
      From worker 2:    Stacktrace:
      From worker 2:     [1] getproperty(::Module, ::Symbol) at .\Base.jl:26
      From worker 2:     [2] top-level scope at C:\Users\mthel\.julia\packages\PooledArrays\4I2sD\src\PooledArrays.jl:162  
      From worker 2:     [3] include(::Function, ::Module, ::String) at .\Base.jl:380
      From worker 2:     [4] include(::Module, ::String) at .\Base.jl:368
      From worker 2:     [5] top-level scope at none:2
      From worker 2:     [6] eval at .\boot.jl:331 [inlined]
      From worker 2:     [7] eval(::Expr) at .\client.jl:467
      From worker 2:     [8] top-level scope at .\none:3
      From worker 2:    in expression starting at C:\Users\mthel\.julia\packages\PooledArrays\4I2sD\src\PooledArrays.jl:162
      From worker 2:    ERROR: LoadError: Failed to precompile PooledArrays [2dfb63ee-cc39-5dd5-95bd-886bf059d720] to C:\Users\mthel\.julia\compiled\v1.5\PooledArrays\vi11X_tAWrN.ji.
      From worker 2:    Stacktrace:
      From worker 2:     [1] error(::String) at .\error.jl:33
      From worker 2:     [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1305
      From worker 2:     [3] _require(::Base.PkgId) at .\loading.jl:1030
      From worker 2:     [4] require(::Base.PkgId) at .\loading.jl:928
      From worker 2:     [5] require(::Module, ::Symbol) at .\loading.jl:923
      From worker 2:     [6] include(::Function, ::Module, ::String) at .\Base.jl:380
      From worker 2:     [7] include(::Module, ::String) at .\Base.jl:368
      From worker 2:     [8] top-level scope at none:2
      From worker 2:     [9] eval at .\boot.jl:331 [inlined]
      From worker 2:     [10] eval(::Expr) at .\client.jl:467
      From worker 2:     [11] top-level scope at .\none:3
      From worker 2:    in expression starting at C:\Users\mthel\.julia\packages\CSV\YUbbG\src\CSV.jl:15

Out put of versioninfo():

Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: AMD Ryzen 9 3950X 16-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, znver2)
Environment:
  JULIA_EDITOR = "C:\Users\mthel\AppData\Local\Programs\Microsoft VS Code\Code.exe"
  JULIA_NUM_THREADS = 32

CSV version is [336ed68f] CSV v0.8.3 and Pluto is [c3e4b0f8] Pluto v0.12.20.

Other users are experiencing this error as well: https://discourse.julialang.org/t/failed-to-precompile-csv-in-pluto/54633/4

@mthelm85
Copy link
Contributor Author

mthelm85 commented Feb 5, 2021

A Discourse user @thatcher was able to get everything working by using the following package versions:

(@v1.5) pkg> rm CSV
(@v1.5) pkg> rm DataFrames
(@v1.5) pkg> rm PooledArrays
(@v1.5) pkg>  rm CategoricalArrays
(@v1.5) pkg>  add CSV@0.8.2
(@v1.5) pkg>  add CategoricalArrays@0.9.0
(@v1.5) pkg>  add DataFrames@0.22.3
(@v1.5) pkg>  add PooledArrays@0.5.3

@RobvanW
Copy link

RobvanW commented Feb 5, 2021

Just to confirm from my side:

Funny thing is CSV (0.8.3), DataFramesMeta (0.60) and DataFrames (0.18.3) load & precompile nicely when run from the REPL include("pluto_notebook_file.jl") but all fail to precompile in the Pluto notebook environment.

Since it doesn't happen to any other packages, well at the moment at least, I wonder if the bug is in Pluto or in these packages (and other packages might follow suit).

@j-fu
Copy link
Contributor

j-fu commented Feb 6, 2021

See the discussion under JuliaData/CategoricalArrays.jl#325 :

It seems that it is somehow linked to the way Pkg and Pluto interact.

Somehow under Pluto, an older version 1.4.x of DataAPI.jl was drawn in, though both DataFrames.jl (via PooledArrays.jl) and CategoricalArrays.jl have requirements for 1.5 in their Project.toml .

I was able to fix my situation by removing .julia/packages and .julia/environments, and re-adding/re-instantiating.

@bkamins
Copy link

bkamins commented Feb 6, 2021

Somehow under Pluto, an older version 1.4.x of DataAPI.jl was drawn in, though both DataFrames.jl (via PooledArrays.jl) and CategoricalArrays.jl have requirements for 1.5 in their Project.toml .

It might be a problem on package server side - hopefully now it is resolved (see https://discourse.julialang.org/t/pkg-protocol-delay-in-updating-generalregistry/54521 for a discussion of similar problems).

@fonsp
Copy link
Owner

fonsp commented Feb 8, 2021

A possible cause is that Pluto loads Tables.jl on the notebook process before running your code, so the loaded version of Tables could be different from the one resolved by Pkg.

Can you use this inside the notebook to check which version of Tables was loaded? It should be >=1.1

@juliohm
Copy link

juliohm commented Feb 18, 2021

I am having the same issue on Manjaro Linux. The same error with PooledArrays.jl inside a Pluto session.

@fonsp fonsp closed this as completed in 6800da3 Feb 20, 2021
@fonsp
Copy link
Owner

fonsp commented Feb 20, 2021

It was a problem on the Pluto side (#895 (comment) ), and I fixed it on master 🌈

We will backport to a patch release for 0.12 soon

fonsp added a commit that referenced this issue Feb 20, 2021
@fonsp fonsp mentioned this issue Feb 20, 2021
66 tasks
@filchristou
Copy link

I get the same problem for the ReinforcementLearning package. (works with REPL)
I am using Pluto v0.14.8.
pluto_failed2precomp

@filchristou
Copy link

Okey. After deleting, removing, removing, installing, removing deleting, installing it worked. sorry for the hasty comment but again it looks like something is not working 100% .. Also I observed that the compilation time is much more that REPL's. idk if this is normal.

@adomasbaliuka
Copy link

I have the same error as OP again, on Windows. The problem is NOT solved by deleting the .julia folder or reinstalling everything. The CSV package works fine outside of Pluto.

Failed to precompile CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b] to C:\Users\MyName\.julia\compiled\v1.7\CSV\jl_BB08.tmp.

error(::String)@error.jl:33
compilecache(::Base.PkgId, ::String, ::IO, ::IO, ::Bool)@loading.jl:1466
compilecache(::Base.PkgId, ::String)@loading.jl:1410
_require(::Base.PkgId)@loading.jl:1120
require(::Base.PkgId)@loading.jl:1013
require(::Module, ::Symbol)@loading.jl:997
top-level scope@[Local: 9](http://localhost:1234/edit?id=c5df97f0-c49c-11ec-23db-83e8fcd277e9#)

Further stacktrace info and versions of stuff here: error_details.txt.

@fonsp
Copy link
Owner

fonsp commented Apr 25, 2022

@adomasbaliuka Can you share a notebook file that does not work?

@fonsp fonsp reopened this Apr 26, 2022
@adomasbaliuka
Copy link

Thanks for the quick response and sorry, it seems I didn't test this carefully enough. The problem is with my notebook reproduce_CSV_compile_bug.jl.txt, though I don't know what the problem is exactly. Maybe some package versions contained in the internal manifest are inconsistent.

If I make a new notebook, even with the same list of packages to use, it actually works fine.

Is there a way to "order a given Pluto notebook to regenerate its internal manifest" or something?

@fonsp
Copy link
Owner

fonsp commented Apr 26, 2022

Hey @adomasbaliuka ! In that case, it looks like your notebook was still using a broken version of CSV.jl.

You can update package by clicking on the checkmark next to a package name, and then the update button. More info here: https://github.com/fonsp/Pluto.jl/wiki/%F0%9F%8E%81-Package-management

@fonsp fonsp closed this as completed Apr 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants