Can't use @load within a module

[petrostat13]

I am trying to load the SVMRegressor model within a module. Simply using the @load macro just as shown below, doesn't seem to work.

module Load
using MLJ

@load SVMRegressor

end

The error message I get is this:

WARNING: eval into closed module MLJBase:
** incremental compilation may be fatally broken for this module **

Even if I try to replicate the macro using the __init__() method, I still can't load it.

module Load
using MLJ
import MLJModels
import ScikitLearn
import MLJModels.ScikitLearn_.SVMRegressor

function __init__()
    try
        SVMRegressor()
    catch
        try # hack for baremodules that have imported Base.eval:
            SVMRegressor = Base.SVMRegressor
            SVMRegressor()
        catch
            @warn "Code is loaded but no instance returned. "
            nothing
        end
    end
    nothing
end

end

I always get

ERROR: LoadError: UndefVarError: ScikitLearn_ not defined

I also tried using

MLJModels.load("SVMRegressor")

and

MLJModels.load("SVMRegressor", modl=Main.Load)

but it doesn't work either.

I really need to load it within a module, so any help would be greatly appreciated.
Thank you!

[ablaom]

Related: JuliaAI/MLJModels.jl#22, which has a work-around which may or may not help.

[ablaom]

@petrostat13 Thanks for posting!

I am not able to reproduce. Perhaps you may be using an old version of MLJModels? In older versions the package metadata was loaded in an init function which is known to cause this kind of issue.

julia> using MLJ

julia> module Load
       using MLJ

       @load SVMRegressor

       end
Main.Load

julia> using Pkg; Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 25 entries:
  "CSV"                    => v"0.5.14"
  "MLJ"                    => v"0.5.2"
  "Coverage"               => v"0.9.3"
  "Distributions"          => v"0.21.5"
  "Revise"                 => v"2.2.2"
  "NearestNeighbors"       => v"0.4.3"
  "MLJLinearModels"        => v"0.2.0"
  "OrderedCollections"     => v"1.1.0"
  "GaussianProcesses"      => v"0.10.0"
  "Missings"               => v"0.4.3"
  "ScikitLearn"            => v"0.5.1"
  "ScientificTypes"        => v"0.2.3"
  "MLJBase"                => v"0.7.3"
  "ColorTypes"             => v"0.8.0"
  "MultivariateStats"      => v"0.7.0"
  "DecisionTree"           => v"0.9.1"
  "RDatasets"              => v"0.6.4"
  "CategoricalArrays"      => v"0.7.1"
  "Tables"                 => v"0.2.11"
  "ComputationalResources" => v"0.3.0"
  "MLJModels"              => v"0.5.5"
  "DataFrames"             => v"0.19.4"
  "LIBSVM"                 => v"0.3.1"
  "GLM"                    => v"1.3.3"
  "MLBase"                 => v"0.8.0"

julia> versioninfo()
Julia Version 1.1.1
Commit 55e36cc (2019-05-16 04:10 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin15.6.0)
  CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_PATH = /Applications/Julia-1.1.app/Contents/Resources/julia/bin/julia

julia> 

If updating does not fix your problem, can you please post a copy of your environment's Manifest.toml file (or do Pkg.installed(), as above?

[ablaom]

Addendum: While loading from a module should work, loading from a module that is part of a package may not JuliaAI/MLJModels.jl#22 but there is a work-around.

[petrostat13]

@ablaom Thanks for the reply!

I saw the linked issue but I still couldn't load SVMRegressor. I updated MLJ (and MLJModels) but the problem persisted.

I can normally compile the module from the REPL just like you, but when I build it from file I get the error:

ERROR: LoadError: LoadError: UndefVarError: ScikitLearn_ not defined

Here is my installed packages

julia> Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 23 entries:
  "CSV"               => v"0.5.14"
  "MLJ"               => v"0.5.2"
  "TableView"         => v"0.4.1"
  "Distributions"     => v"0.21.5"
  "Atom"              => v"0.11.3"
  "NearestNeighbors"  => v"0.4.3"
  "Juno"              => v"0.7.2"
  "Conda"             => v"1.3.0"
  "ScikitLearn"       => v"0.5.1"
  "ScientificTypes"   => v"0.2.3"
  "PyCall"            => v"1.91.2"
  "MLJBase"           => v"0.7.5"
  "StatsBase"         => v"0.32.0"
  "MultivariateStats" => v"0.7.0"
  "Tables"            => v"0.2.11"
  "DecisionTree"      => v"0.9.1"
  "Flux"              => v"0.9.0"
  "Plots"             => v"0.27.0"
  "XGBoost"           => v"0.4.2"
  "ODBC"              => v"0.8.1"
  "MLJModels"         => v"0.5.6"
  "DataFrames"        => v"0.19.4"
  "GLM"               => v"1.3.3"

[ablaom]

Thanks for the update. As mentioned in the other thread, I expect this is really an issue with Requires.jl. So the next step is to see if we can build a MWE showing this is true. Is this something you're happy to explore yourself? I don't see myself getting to this for many weeks given other project priorities.

[petrostat13]

I just tried

__precompile__(false)

module Load
using MLJ

@load SVMRegressor

end

and it worked.

I will also try to figure out the problem with Requires.jl, but my experience as a developer is practically zero, so I cannot promise a lot ;)
Thanks again for the help!

[ablaom]

Well, you already made significant progress!

[ablaom]

Will be resolved in MLJModels 0.14 and whence in next release of MLJ