Project Report: Deep Learning for Pseudo-random Generation using an Autoencoder

1. Project Goal and Approach

The primary goal of this project was to explore the use of deep learning, specifically an autoencoder, for generating pseudo-random data that exhibits similar characteristics to a given training dataset. The approach involved training an autoencoder to learn a compressed representation (latent space) of the input data. Pseudo-random data is then generated by sampling points from this learned latent space and decoding them using the trained decoder model.

2. Data Source and Preprocessing

• Data Source: The Fashion MNIST dataset was used for this project. This dataset consists of 70,000 grayscale images (60,000 for training and 10,000 for testing) of fashion items, each 28x28 pixels. The dataset was readily available within the TensorFlow/Keras library, eliminating the need for manual download from external sources like Kaggle.
• Preprocessing: The data was preprocessed by:
  • Loading the training and testing sets.
  • Normalizing the pixel values from the range [0, 255] to [0, 1] by dividing by 255.0.
  • Reshaping the 28x28 images into flattened vectors of size 784 to be compatible with the fully connected layers of the autoencoder.

3. Autoencoder Architecture and Implementation

• Architecture: A simple feedforward autoencoder architecture was designed with an encoder and a decoder.
  • Encoder: The encoder consisted of dense layers with ReLU activation, progressively reducing the dimensionality of the input data to a latent space. The layers had 128, 64, and 32 units, respectively, with the final layer representing the latent space (latent_dim = 32).
  • Decoder: The decoder mirrored the encoder structure but in reverse, using dense layers with ReLU activation, except for the output layer which used a sigmoid activation to produce pixel values between 0 and 1. The layers had 64, 128, and 784 units, respectively.
• Implementation: The autoencoder was implemented using the TensorFlow/Keras library. The Model class was used to define the autoencoder, encoder, and decoder as separate models for flexibility. The autoencoder was compiled with the Adam optimizer and binary cross-entropy as the loss function, suitable for pixel value reconstruction.

4. Model Training

The autoencoder model was trained on the preprocessed Fashion MNIST training data (x_train) with the goal of reconstructing the input. The model was trained for 20 epochs with a batch size of 128. The training progress was monitored using the validation loss on the testing data (x_test). The training process showed a decrease in both training and validation loss, indicating that the model was learning to effectively compress and reconstruct the image data.

5. Pseudo-random Generation

After training, the decoder model was used to generate pseudo-random data. This was achieved by:

• Generating random points in the latent space by sampling from a normal distribution with the same dimensionality as the latent space (32).
• Passing these random latent vectors through the trained decoder model to generate new data points in the original data space (784-dimensional vectors).
• Reshaping the generated data back into the image format (28x28 pixels) for visualization.

A set of 10 pseudo-random images were generated and displayed, showing variations of fashion items that the autoencoder learned to represent.

6. Output Generation

The project successfully generated 100 pseudo-random data points. These were generated as 784-dimensional vectors and then reshaped into 28x28 image format, resulting in a final output shape of (100, 28, 28).

7. Analysis of Results

The generated pseudo-random images qualitatively resemble the fashion items from the training dataset, suggesting that the autoencoder successfully learned the underlying distribution of the data in the latent space. The variations in the generated images demonstrate the model's ability to create new data points by exploring this latent space.

8. Project Organization

The project was organized into three Python files:

• data_preprocessing.py: Contains the functions for loading and preprocessing the Fashion MNIST dataset.
• autoencoder_model.py: Contains the functions for building, compiling, training, and saving the autoencoder, encoder, and decoder models.
• generate_random_data.py: Contains the functions for loading the trained decoder model, generating pseudo-random data from the latent space, and visualizing the generated output.

This organization promotes modularity and reusability of the code components.

9. Conclusion and Future Work

This project successfully demonstrated the use of an autoencoder for pseudo-random data generation. The trained model can generate new data instances that are similar to the training data.

Future work could include:

• Implementing a quantitative evaluation metric to assess the diversity and quality of the generated data.
• Experimenting with different autoencoder architectures (e.g., convolutional autoencoders for image data) and hyperparameters to potentially improve the quality of the generated data.
• Exploring the use of Variational Autoencoders (VAEs), which are specifically designed for generative tasks and provide a more structured latent space for sampling.