Autoencoders as a dimensionality reduction algorithm

[jeremymanning]

We could support dimensionality reduction through autoencoders. Here's a useful looking tutorial (it looks relatively straightforward to implement all of the variants in the tutorial): https://blog.keras.io/building-autoencoders-in-keras.html

• shallow autoencoder
• deep autoencoder
• sparse autoencoder
• convolutional autoencoder
• sequence-to-sequence autoencoder (good for timeseries data)
• variational autoencoder

[jeremymanning]

If we implement this using the standard hypertools API (e.g. through the reduce function/flag), it would provide a really convenient way to train and apply autoencoders. I think some neat demos would come out of it.

[jeremymanning]

Status in the 1.0 update (PR #272 #272): no change — grep -rli "keras|tensorflow|torch\b" hypertools/ still only turns up hypertools/predict/chronos.py (a pretrained forecasting transformer, unrelated to dimensionality reduction). None of the 6 requested autoencoder variants (shallow/deep/sparse/convolutional/seq2seq/variational) exist. Notably, torch is now already installed transitively in this environment (via pydata-wrangler[hf]/chronos-forecasting's predict-hf extra), which lowers the practical barrier versus 2017 when this was filed.

How this could be built on the 1.0 architecture: Add hypertools/reduce/autoencoder.py implementing an sklearn-compatible class (constructor + fit/transform/fit_transform/inverse_transform) wrapping a small torch.nn.Module, then register it (e.g. 'Autoencoder', 'DeepAutoencoder') in hypertools/reduce/reduce.py's models dict so it's picked up automatically by both reduce() and core/model.py's merged registry (apply_model). Gate torch behind a new optional extra in pyproject.toml (mirroring the existing predict-hf = ["chronos-forecasting>=2.0"] pattern) so the base install stays light. Sparse/convolutional/seq2seq/VAE variants would each need distinct network architectures and loss functions — realistically separate follow-up PRs, not one PR.

Alternatives / prior art:

• No well-maintained, sklearn-fit_transform-compatible autoencoder package exists on PyPI today that would drop into the registry without custom wrapping (checked; nothing comparable to how umap-learn/sklearn plug in directly).
• umap-learn ships ParametricUMAP, a neural (encoder-based) alternative that supports out-of-sample .transform() — but it depends on TensorFlow/Keras, a second heavy DL framework hypertools doesn't currently use (torch is used instead for chronos), so adding it would mean supporting two DL backends.
• For sequence/timeseries data specifically, hypertools's own hyp.predict (Chronos, a pretrained transformer) already covers a chunk of the "seq2seq autoencoder for timeseries" motivation from a forecasting angle, though not as a reusable dimensionality-reduction embedding.

Note: parts of this ask are dated — the 2017 Keras-blog-tutorial framing (shallow/deep/sparse/convolutional variants) reflects pre-UMAP/pre-transformer practice; today, UMAP (already supported, including a transform()-capable model per issue #161) and PCA/t-SNE cover most of the "nonlinear embedding" use cases this issue targeted. A VAE variant remains the one item with no existing hypertools equivalent.

Tagging as high effort (full 6-variant ask, incl. new optional dependency, tests, docs); a shallow+deep-only MVP would be medium effort. Leaving open for consideration in a future release.

[jeremymanning]

This needs more research. But in general:

• torch should be an optional extra
• we need a nice wrapper for easily creating autoencoders, fitting them, and applying them to existing data and/or new data (if model is returned)
• all of the autoencoder models described in the initial issue should be supported, and these should be integrated into the reduce module. it's probably worth splitting off an autoencoders.py that implements these.

we should use modern libraries and current best practices