intrinsic-dimension-projections

Jonathan Lam & Richard Lee

See the paper and the presentation associated with this research.

keras_ext: main TF2/keras low-level architectural files (projection implementation)
models: model architectures
plots: sample plots used in report (scripts to generate these plots are not included)
runs: sample pickled runs (not a complete list of models used in the report)
scripts: scripts to automate model building and training

The primary model for training was MNIST, and the training loop is in scripts/mnist.py. There, you can adjust the model hyperparameters:

epochs: maximum number of epochs before termination (ignoring early stopping)
intrinsic_dims: iterable of intrinsic dimension sizes to train
initializers: initializers for the initial weight matrices theta_0
lrs: set of learning rates to train with
model_types: list of model types to train; can choose from "linear", "power", and "rff"
normalize_p: whether to normalize the output basis vectors in the projection matrix

Apply other projection types (other than random linear mappings); in particular, try random fourier features (RFF) and power terms
Rewrite/upgrade/modernize to Python3/TF2 (e.g., type hinting for better IDE support, do more linting)
Include better documentation/explanations of terms than original code

Placing a little bit of extra emphasis here because the codebase from the previous project was a little hard to decipher:

Attempt to adhere to PEP8 as best as possible (use IntelliJ linting/code refactor when possible)
Type hints in parameter lists/other initializations when possible to allow for maximum automated inferencing
Comment at beginning of each file to describe what the file is and defining relevant terms
Clear variable names (clarity at the cost of longer variable names)
Comments describing every function (except obvious cases and some overloaded methods) for maximum clarity

Navigation Menu