Varying STLSQ parameter gives interesting error behaviour

[albheim]

Noticed I could make julia throw different errors just by changing the parameter for STLSQ, and neither error was very descriptive. The second error is accompanied by a warning which does tell me that the regression failed, and I assume that is why the error comes after, though it seems odd to warn about a thing if it is just going to error later.

Haven't looked into the reason at all, but here is an MWE that I tested on julia 1.7.2 with DataDrivenDiffEq v0.8.4

julia> using DataDrivenDiffEq, Symbolics

julia> x = reshape(range(-1, 1, length=300), 3, 100)
3×100 reshape(::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, 3, 100) with eltype Float64:
 -1.0       -0.979933  -0.959866  -0.939799  -0.919732  -0.899666  -0.879599  -0.859532  …  0.846154  0.866221  0.886288  0.906355  0.926421  0.946488  0.966555  0.986622
 -0.993311  -0.973244  -0.953177  -0.93311   -0.913043  -0.892977  -0.87291   -0.852843     0.852843  0.87291   0.892977  0.913043  0.93311   0.953177  0.973244  0.993311
 -0.986622  -0.966555  -0.946488  -0.926421  -0.906355  -0.886288  -0.866221  -0.846154     0.859532  0.879599  0.899666  0.919732  0.939799  0.959866  0.979933  1.0

julia> y = [x[1:1, :] .* x[2:2, :]; sin.(x[3:3, :])]
2×100 Matrix{Float64}:
  0.993311   0.953714   0.914923   0.876936   0.839756   0.80338   0.76781    0.733045  …  0.721636  0.756132  0.791434  0.827541  0.864453  0.902171  0.940694  0.980023
 -0.834168  -0.822933  -0.811368  -0.799475  -0.787261  -0.77473  -0.761887  -0.748736     0.757537  0.770483  0.783119  0.795439  0.80744   0.819115  0.83046   0.841471

julia> prob = DirectDataDrivenProblem(x, y)
Direct DataDrivenProblem{Float64} ##DDProblem#1482 in 3 dimensions and 100 samples

julia> @variables x[1:3]
1-element Vector{Symbolics.Arr{Num, 1}}:
 x[1:3]

julia> basis = Basis(polynomial_basis(x, 2), x)
Model ##Basis#1483 with 10 equations
States (3):
  x[1]
  x[2]
  x[3]
Parameters (0):

julia> res = solve(prob, basis, STLSQ(0.1)) # Works
Linear Solution

julia> res = solve(prob, basis, STLSQ(0.2)) # Small increase gives warning + error
ERROR: BoundsError: attempt to access 1-element Vector{Num} at index [2]
Stacktrace:
 [1] getindex
   @ ./array.jl:861 [inlined]
 [2] construct_basis(X::Matrix{Float64}, b::Basis, implicits::Vector{Any}; dt::Float64, lhs::Symbol, is_implicit::Bool, eval_expression::Bool)
   @ DataDrivenDiffEq ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solution.jl:286
 [3] DataDrivenSolution(prob::DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, s::DataDrivenDiffEq.SparseLinearSolution{Array{Float64, 3}, Matrix{Float64}, Tuple{Vector{UnitRange{Int64}}, UnitRange{Int64}}, Vector{Float64}, Matrix{Float64}, STLSQ{Float64}, DataDrivenCommonOptions{Float64, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, b::Basis, opt::STLSQ{Float64}, implicits::Vector{Any}; eval_expression::Bool, digits::Int64, by::Symbol, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ DataDrivenDiffEq ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solution.jl:359
 [4] solve!(p::DataDrivenDiffEq.SparseIdentificationProblem{Array{Float64, 3}, DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, Basis, Vector{UnitRange{Int64}}, UnitRange{Int64}, STLSQ{Float64}, DataDrivenCommonOptions{Float64, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}})
   @ DataDrivenDiffEq ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solve/sparse_identification.jl:131
 [5] solve(::DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ CommonSolve ~/.julia/packages/CommonSolve/TGRvG/src/CommonSolve.jl:23
 [6] solve(::DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, ::Basis, ::STLSQ{Float64})
   @ CommonSolve ~/.julia/packages/CommonSolve/TGRvG/src/CommonSolve.jl:23
 [7] top-level scope
   @ REPL[1886]:1

julia> res = solve(prob, basis, STLSQ(0.4)) # Another small increase gives a different error
┌ Warning: Sparse regression failed! All coefficients are zero.
└ @ DataDrivenDiffEq ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solution.jl:351
ERROR: MethodError: no method matching DataDrivenSolution(::Basis, ::Vector{Any}, ::Symbol, ::STLSQ{Float64}, ::Matrix{Float64}, ::DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct})
Closest candidates are:
  DataDrivenSolution(::DataDrivenDiffEq.AbstractBasis, ::AbstractVector, ::Symbol, ::A, ::O, ::DataDrivenDiffEq.AbstractDataDrivenProblem, ::Bool; eval_expression, kwargs...) where {A, O} at ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solution.jl:62
  DataDrivenSolution(::DataDrivenDiffEq.AbstractBasis, ::AbstractVector, ::Symbol, ::Any, ::Any, ::DataDrivenDiffEq.AbstractDataDrivenProblem, ::AbstractVector, ::AbstractVector, ::AbstractVector; kwargs...) at ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solution.jl:53
  DataDrivenSolution(::Any, ::Any, ::B, ::O, ::Any; eval_expression, digits, by, kwargs...) where {B<:DataDrivenDiffEq.AbstractBasis, O<:DataDrivenDiffEq.AbstractOptimizer} at ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solution.jl:346
  ...
Stacktrace:
 [1] DataDrivenSolution(prob::DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, s::DataDrivenDiffEq.SparseLinearSolution{Array{Float64, 3}, Matrix{Float64}, Tuple{Vector{UnitRange{Int64}}, UnitRange{Int64}}, Vector{Float64}, Matrix{Float64}, STLSQ{Float64}, DataDrivenCommonOptions{Float64, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, b::Basis, opt::STLSQ{Float64}, implicits::Vector{Any}; eval_expression::Bool, digits::Int64, by::Symbol, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ DataDrivenDiffEq ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solution.jl:352
 [2] solve!(p::DataDrivenDiffEq.SparseIdentificationProblem{Array{Float64, 3}, DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, Basis, Vector{UnitRange{Int64}}, UnitRange{Int64}, STLSQ{Float64}, DataDrivenCommonOptions{Float64, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}})
   @ DataDrivenDiffEq ~/.julia/packages/DataDrivenDiffEq/16LV4/src/solve/sparse_identification.jl:131
 [3] solve(::DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ CommonSolve ~/.julia/packages/CommonSolve/TGRvG/src/CommonSolve.jl:23
 [4] solve(::DataDrivenProblem{Float64, true, DataDrivenDiffEq.Direct}, ::Basis, ::STLSQ{Float64})
   @ CommonSolve ~/.julia/packages/CommonSolve/TGRvG/src/CommonSolve.jl:23
 [5] top-level scope
   @ REPL[1888]:1

[AlCap23]

Thanks for the post!

The second is expected and should be handled differently. This means all coefficients are zero and no right answer is found.

The first one needs investigating a little :)

[AlCap23]

Found the bug and will fix it over the day 👍

[albheim]

Nice!

Yeah, the second error was okay since you got the warning first.

To me it just felt a little strange to not error with the message that no solution was found, and instead warn about it just to later error with a much harder to understand message. But there might be a reason to do this, not complaining. Just wanted to make you aware of what I ran into :)