Skip to content
Machine Learning Fashion Modelling. Training a Variational Autoenconder (VAE) on Keras TensorflowJS framework achieving results on highly challenging dataset of over 60,000 fashion MNIST images.
JavaScript HTML Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Ms.Robot Fashion Modelling 👩🏻‍🔬

* ✿ ❀ Training a variational autoencoder on the Fashion MNIST dataset ❀ ✿ *

Status GitHub Issues GitHub Pull Requests License

Table_of_Contents 💜

Motivation 💜

  • Train a variational autoenconder (VAE) using TensorFlow.js on Node with technical requirements:

    • Tensorflow==1.12.0
    • Keras==2.2.4
    • TensorflowJS==0.6.7
  • The model will be trained on the Fashion MNIST dataset

    Gender/Brilliance bias

    Image of Gender/Brilliance Bias. Google's suggestions when user types "How to get my daughter into... " Reference Cameron Russell video "Looks aren't everything. Believe me, I'm a model." Link to video here.

Autoencoders 💜

Autoencoders yay

Image. How autoencoders work using the MNIST data set with the number "2"

  • "Autoencoding" == Data compression algorithm with compression and decompression functions
  • User defines the parameters in the function using variational autoencoder
  • Self-supervised learning where target models are generated from input data
  • Implemented with neural networks - useful for problems in unsupervised learning (no labels)

Variational Autoencoders (VAE) 💜

  • Variational autoencoders are autoencoders, but with more constraints

  • Generative model with parameters of a probability distribution modeling the data

  • The encoder, decoder, and VAE are 3 models that share weights. After training the VAE model, the encoder can be used to generate latent vectors

  • Refer to Keras tutorial for variational autoenconder (MNIST digits) except we will be using Fashion data instead :)

    Spark Joy!!!

    Image. Marie Kondo sparking joy with the wonders of variational autoencoders 👩🏻‍🔬

Variational Autoencoder (VAE) Example 💜

Example of encoder network maping inputs to latent vectors:

  • Input samples x into two parameters in latent space = z_mean and z_log_sigma
  • Randomly sample points z from latent normal distribution to generate data
  • z = z_mean + exp(z_log_sigma) * epsilon, where epsilon is a random normal tensor
  • Decoder network maps latent space points back to the original input data
 x = Input(batch_shape=(batch_size, original_dim))
 h = Dense(intermediate_dim, activation='relu')(x)
 z_mean = Dense(latent_dim)(h)
 z_log_sigma = Dense(latent_dim)(h)

Sample Code for VAE encoder network

Label_descriptions 💜

Ms.Robot has the following fashion pieces in her wardrobe:

  1. T-shirt/top
  2. Trouser
  3. Pullover
  4. Dress
  5. Coat
  6. Sandal
  7. Shirt
  8. Sneaker
  9. Bag
  10. Ankle boot

Plot of subset Images from Fashion MNIST dataset

Image. The 0 to 9 label descriptions for the Fashion MNIST dataset

Prepare_the_node_environment 💜

# Or
npm install

Download_the_fashion_data 💜

  • Download Ms.Robot's fashion dataset with over 60,000 fashion training set images train-images-idx3-ubyte.gz from here
  • Uncompress the large file size (26 MBytes)
  • Move the uncompressed file train-images-idx3-ubyte into dataset folder in the example folder of this repo

Run_the_training_script 💜

  • Can not feed all the data to the model at once due to computer memory limitations so data is split into "batches"
  • When all batches are fed exactly once, an "epoch" is completed. As training script runs, preview images afer every epoch will show
  • At the end of each epoch the preview image should look more and more like an item of clothing for Ms.Robot
yarn train

Loss_error_function 💜

  • Loss function to account for error in training since Ms.Robot is picky about her fashion pieces

  • Two loss function options: The default binary cross entropy (BCE) or mean squared error (MSE)

  • The loss from a good training run will be approx 40-50 range whereas an average training run will be close to zero

    Example loss curve from training

    Image of loss curve with the binary cross entropy error function

TensorBoard_monitoring_model_training 💜

Use --logDir flag of yarn train command. Log the batch-by-batch loss values to a log directory

yarn train --logDir /tmp/vae_logs

Start TensorBoard in a separate terminal to print an http:// URL to the console. The training process can then be monitored in the browser by Ms.Robot:

pip install tensorboard 
tensorboard --logdir /tmp/vae_logs

Tensorboard Monitoring

Image. Tensorboard's monitoring interface.


Image. Tensorboard's monitoring interface.

Conclusion_Model_Discussion 💜

Results show a generative model with parameters of a probability distribution variational autoencoder (VAE) is capable of achieving results on a highly challenging dataset of over 60,000 fashion set images using machine learning. The variational autoencoder VAE is a generative model which means it can be used to generate new fashion pieces for Ms.Robot. This is done by scanning the latent plane, sampling the latent points at regular intervals, to generate the corresponding fashion piece for each point. Run to serve the model and the training web page.

yarn watch

screenshot of results on fashion MNIST. A 30x30 grid of small images

Image of completed training results on fashion MNIST 30x30 grid of small images for Ms.Robot. Visualization of the latent manifold* that was "generated" by the Ms.Robot generative model.

References 💜

You can’t perform that action at this time.