interactions in random effect specification

[nathanieldaw]

One feature that works in lme4 that seems not to work in MixedModels is interactions in the random effects. In MixedModels, a formula like

a ~ b * c + (b*c | subject)

fails with MethodError: no method matching getindex(::DataFrames.DataFrame, ::Expr)

and if I expand the * manually as b + c + b&c then it is the last term that triggers the error.

The problem occurs in the call to ModelFrame, but I have to admit I don't don't really understand what's going on in those functions.

Anyway the obvious workaround of constructing b .* c works, but I would appreciate any help getting the less cumbersome syntax for this working.

[dmbates]

Unfortunately, issues like this are in a backlog waiting for changes like JuliaStats/StatsModels.jl#30 in https://github.com/JuliaStats/StatsModels.jl which, in turn are waiting on the release of v0.11.0 of https://github.com/JuliaStats/DataFrames.jl

Right now everything is in a holding pattern waiting for this version of DataFrames using Nulls and CategoricalArrays but I don't see any progress on e.g. JuliaData/DataFrames.jl#1232. I have no idea when, or even if, these changes will take place. I have asked if there is a roadmap for these changes but have not received any replies.

[nalimilan]

Where did you ask? The roadmap is https://github.com/JuliaData/DataFrames.jl/issues?q=is%3Aopen+is%3Aissue+milestone%3A0.11, and there's at least one blocking issue there which is going to be resolved soon.

[yanivabir]

Is there any news on this? DataFrames v0.11.0 has been released and JuliaStats/StatsModels.jl#30 seems to have merged. Being able to fit random slopes for interactions is a very important feature..

[kleinschmidt]

Fitting random slopes is possible on the current release of StatsModels if you specify them directly, as in (1+a+b+a&b | subject). On StatsModels master the * syntax works, so you can do (1+a*b | subject) to get the same specification (due to JuliaStats/StatsModels.jl#54). We can tag a release once @dmbates confirms that it won't break anything else here.

[dmbates]

Looks good for tagging a release of StatsModels. Thanks for the work on that, Dave.

I have a release of MixedModels with the @eval(@formula($nothing ~ $l)) fix ready to go once the new release of StatsModels is up.

[getzze]

Julia 1.0.3
MixedModels v1.1.2
DataFrames v0.16.0
StatsModels v0.3.1

There is still a problem with interactions in the random effects.
From example 6 from the documentation:

fm6 = fit(LinearMixedModel, @formula(Y ~ 1 + A * I + (1|G) + (1|H)), dat[:InstEval])

Changing the formula leads to errors:

fm6 = fit(LinearMixedModel, @formula(Y ~ 1 + A * I + (1 + A|G) + (1|H)), df)
>>> MethodError: no method matching rmul!(::SparseArrays.SparseMatrixCSC{Float64,Int32}, ::RepeatedBlockDiagonal{Float64,LinearAlgebra.LowerTriangular{Float64,Array{Float64,2}}})

fm6 = fit(LinearMixedModel, @formula(Y ~ 1 + A * I + (1 + A|G)), df)
>>> OK

fm6 = fit(LinearMixedModel, @formula(Y ~ 1 + A * I + (1 + A & I|G)), df)
>>> MethodError: no method matching getindex(::DataFrame, ::Expr)

fm6 = fit(LinearMixedModel, @formula(Y ~ 1 + A * I + (1|G & H)), df)
>>> MethodError: no method matching getindex(::DataFrame, ::Expr)