Missing NonlinearENA

[snotskie]

NonlinearENA was cut since 0.1.0, to allow me time to refactor it to match the 0.5.0 codebase, as well as to put in the much needed graphic design work to make the Nonlinear plots actually look good

[snotskie]

one way I've imagined this is, nonlinear is a layer that adds (1) some metadata, (2) some plots, and (3) some tests. mechanically, it isn't a replacement for a linear rotation, just an intermediate layer for doing other tasks, aka, a "view" of an underlying linear model

it could look something like this:

# Not implemented
model = ENAModel(...)
clustered = NonlinearClusterView(
    model,
    reduceBy=X -> UMAP_(X, ...),
    reduceDimsIn=3, # use first three dims from the model, ignore the rest
    reduceExtraPredictors=[:Day, :Grade => 0.8], # but also include some extra predictors, with optional weighting
    clusterBy=X -> hdbscan(X, ...),
    clusterKey=:ClusterGroup,
    clusterValue=i -> "Group $i"
)
plot(clustered) # will include nonlinear plot in addition to all plots model does
rotated_on_clusters = ENAModel(clustered, rotateBy=MulticlassRotation(:ClusterGroup)
plot(rotated_on_clusters) # will not include nonlinear plot

But I'm not too happy with that format yet

My thinking in any case with a "view" is it:

• is a well-defined pipeline
• is for a well-defined set of tasks
• creates new metadata (the "view" of the model)
  • which is then used in the new plots and tests
    • which are in turn used to accomplish those downstream well-defined tasks
• any operation that can be performed on a model can be performed on a view

[snotskie]

worth noting that this looks a lot like a job for https://github.com/alan-turing-institute/MLJ.jl, besides the particular plotting different views might require, and the wild west that MLJ would create

[snotskie]

talking with folks at icqe23, a simpler syntax would be preferred, eg

pc = pointcloud(model, dims=1:4, format=:wide, metadata=[:Pretest, :Posttest], znorm=true)
um = UMAP_(pc.X, ...)
hdb = hdbscan(transform(um, pc.X), ...)
nl = NonlinearENAModel(model, pc, um, hdb)
plot(nl) # just the nl plot groupedby :ClusterNumber
display(nl) # all the nl specific config and goodness tests, eg. silhouettes
mc = ENAModel(nl, rotateBy=MulticlassRotation(:ClusterNumber))
plot(mc) # just linear plots as-is

this looks a lot more like the python approach, with julia type wrapping to manage ENA integration