Missing package dependency?

[dpsanders]

Describe the bug
Following the README gives an error with a missing package.

To Reproduce
In a clean installation, install MLJ and RDatasets and try to run the code in
https://github.com/alan-turing-institute/MLJ.jl/blob/master/docs/src/index.md up to @load DecisionTreeClassifier.

julia> @load DecisionTreeClassifier
import MLJModels ERROR: LoadError: ArgumentError: Package MLJModels not found in current path:
- Run `import Pkg; Pkg.add("MLJModels")` to install the MLJModels package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:823
 [2] eval at ./boot.jl:328 [inlined]
 [3] eval(::Expr) at ./client.jl:404
 [4] _load(::String, ::String, ::Module) at /Users/dpsanders/.julia/packages/MLJ/HXakA/src/loading.jl:166
 [5] @load(::LineNumberNode, ::Module, ::Any) at /Users/dpsanders/.julia/packages/MLJ/HXakA/src/loading.jl:201
in expression starting at REPL[14]:1

Expected behavior

Either this should work, or the step using MLJModels should be added in the README.

Versions
MLJ v0.2.3

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Environment:
  JULIA_EDITOR = atom

[dpsanders]

Adding MLJModels then gives

julia> @load DecisionTreeClassifier
import MLJModels [ Info: Precompiling MLJModels [d491faf4-2d78-11e9-2867-c94bc002c0b7]
✔
import DecisionTree ERROR: LoadError: ArgumentError: Package DecisionTree not found in current path:
- Run `import Pkg; Pkg.add("DecisionTree")` to install the DecisionTree package.

[dpsanders]

Finally, after installing all the packages I get the following:

julia> fit!(tree, rows=train)
[ Info: Training Machine{DecisionTreeClassifier} @ 1…94.
ERROR: MethodError: no method matching build_tree(::CategoricalArray{String,1,UInt8,String,CategoricalString{UInt8},Union{}}, ::Array{Float64,2}, ::Float64, ::Int64, ::Int64, ::Int64, ::Float64)
Closest candidates are:
  build_tree(::Array{T<:Float64,1}, ::Array{S,2}, ::Any, ::Any, ::Any, ::Any, ::Any; rng) where {S, T<:Float64} at /Users/dpsanders/.julia/packages/DecisionTree/nfB7O/src/regression/main.jl:27
  build_tree(::Array{T,1}, ::Array{S,2}, ::Any, ::Any, ::Any, ::Any, ::Any; rng) where {S, T} at /Users/dpsanders/.julia/packages/DecisionTree/nfB7O/src/classification/main.jl:83
  build_tree(::Array{T<:Float64,1}, ::Array{S,2}, ::Any, ::Any, ::Any, ::Any) where {S, T<:Float64} at /Users/dpsanders/.julia/packages/DecisionTree/nfB7O/src/regression/main.jl:27
  ...
Stacktrace:
 [1] fit(::DecisionTreeClassifier, ::Int64, ::DataFrame, ::CategoricalArray{String,1,UInt8,String,CategoricalString{UInt8},Union{}}) at /Users/dpsanders/.julia/packages/MLJModels/p7doS/src/DecisionTree.jl:110
 [2] #fit!#3(::Array{Int64,1}, ::Int64, ::Bool, ::Function, ::Machine{DecisionTreeClassifier}) at /Users/dpsanders/.julia/packages/MLJ/HXakA/src/machines.jl:131
 [3] (::getfield(StatsBase, Symbol("#kw##fit!")))(::NamedTuple{(:rows,),Tuple{Array{Int64,1}}}, ::typeof(fit!), ::Machine{DecisionTreeClassifier}) at ./none:0
 [4] top-level scope at none:0

[juliohm]

I also share this view that the default installation feels incomplete. Instead of an error I think one could at least ask the user if he desires to install the dependency on the fly.

Regarding the last error, please make sure that you are calling the right version of fit methods. For instance, if you are using the model directly, you should call fit(model, verbosity, X, y). If you are using MLJ machines, then I don't know.

[ablaom]

@dpsanders Can you check the versions of CategoricalArrays in your project manifest? Eg, do ]status -m

The error you report is a known issue unless CategoricalArrays = "<0.5.3". If you update MLJ to latest tagged release, this should go away. If not trying updating MLJBase first.

Let me know if this persists.

[ablaom]

Either this should work, or the step using MLJModels should be added in the README

Already there:

We could make MLJModels an explicit dependency of MLJ, so that even this is unnecessary. The original idea was that external packages implement their own MLJ interface natively; MLJModels is supposed to be a temporary hack. It hasn't worked out that way yet, however.

Regarding this:

import DecisionTree ERROR: LoadError: ArgumentError: Package DecisionTree not found in current path:
- Run `import Pkg; Pkg.add("DecisionTree")` to install the DecisionTree package.

This sounds like an informative message to me. I don't see how an initial "fail" can be avoided, without either: (i) making DecisionTree (and every other external ML pkg) an explicit dependency of MLJ, or (ii) automatically messing with the user's load path (ie Project/Manfest). Neither option is appealing to me. Do you have better suggestion? Or have I misunderstood your objection?

[dpsanders]

The version of CategoricalArrays is 0.5.5.
I installed MLJ yesterday but I see that I don't have the latest tagged version. (Why would that be?)
I just did an ]up and I still don't have the latest version -- not sure what's going on there.

Prompting to install the needed package is a great suggestion.

I guess I was just expecting to be able to copy and paste code from https://alan-turing-institute.github.io/MLJ.jl/dev/ and have it work. Maybe it's just as simple as adding the relevant Pkg commands directly in the "Getting started" text.

Thanks!

[dpsanders]

DataFrames is restricting CategoricalArrays to 0.5.4--0.5.5, and that is not allowing MLJ to update apparently.

[dpsanders]

I finally successfully pinned CategoricalArrays to 0.5.2 and then all the MLJ packages update to the latest version and everything works. Thanks!

[dpsanders]

(I don't see that info in the docs though?)

[ablaom]

Closing as this seems to me a package-management issue, not an MLJ one.

Some extra comments added to documentation in #206 to clarify need add external packages to the user's environment.

[xgdgsc]

Is it because MLJ that my DataFrames is downgraded?

[a93c6f00] ↓ DataFrames v0.19.4 ⇒ v0.18.4
  [1313f7d8] ↓ DataFramesMeta v0.5.0 ⇒ v0.4.1
  [add582a8] ↑ MLJ v0.2.3 ⇒ v0.4.0
  Updating `~/.julia/environments/v1.2/Manifest.toml`
  [324d7699] ↓ CategoricalArrays v0.6.0 ⇒ v0.5.2
  [a93c6f00] ↓ DataFrames v0.19.4 ⇒ v0.18.4
  [1313f7d8] ↓ DataFramesMeta v0.5.0 ⇒ v0.4.1

[ablaom]

I expect the issue is that MLJBase requires CategoricalArrays = "<0.5.3" because of this unresolved issue: JuliaData/CategoricalArrays.jl#199

[xgdgsc]

Thanks.

[ablaom]

You can pin Missings to 0.4.1 to get rid of annoying warning about nonmissingtype.