Pruning Algorithms for Seq2Point Energy Disaggregation
This code repository implements four weight pruning algorithms designed to reduce the size of Zhang et al.'s sequence-to-point deep learning model for use in energy disaggreation / non-intrusive load monitoring. Three alternative network architectures are also available. These include a dropout variant of Zhang et al.'s architecture, a variant with fewer weights and filters, and an architecture implementing both of these.
Numpy, Pandas, and Matplotlib are required for this project:
pip install numpy && pip install pandas && pip install matplotlib
TensorFlow 2.0 GPU is required for GPU-accelerated training and inferencing:
pip install tensorflow-gpu
Lightweight models are produced using the Tensorflow Model Optimisation Toolkit:
pip install tensorflow-model-optimization-toolkit
The REFIT: Electrical Load Measurements Dataset (Cleaned) is required to create training, validation, and testing datasets. This can be downloaded from https://pureportal.strath.ac.uk/en/datasets/refit-electrical-load-measurements-cleaned. Extract the files using 7-Zip and place all the CSV files in a folder titled "refit_dataset" in the root of the project folder.
To download this project, run the following command:
git clone https://github.com/JackBarber98/prunedNILM.git
To generate the required training, testing, and validation datasets for a target appliance, run the following command:
python dataset_generator.py --appliance_name="kettle"
Note that a new dataset must be created for each appliance you wish to train a model for. The following devices are supported:
Structured Probabilistic Pruning
A pruning algorithm proposed by Wang et al. . The probability of a non-specific weight in a layer / convolutional filter being pruned is calculated. A weight mask for each layer is produced using Monte Carlo sampling that, in turn, is used to set insignificant weights to zero.
SPP pruning can be utilised during training using the following:
python train_main.py --pruning_algorithm="spp"
A pruning algorithm proposed by Hur et al. . Gaussian distribution is used to calculate the information gain potential of a specific weight in a layer. By comparing this value to the overall entropy of a layer, weights that have a small value and are less likely to gain information as training continues can be safely removed from the network.
Entropic pruning can be utilised during training using the following:
python train_main.py --pruning_algorithm="entropic"
Relative Threshold Pruning
An implementation of the relative threshold approach to pruning devised by Ashouri et al. . The value of the n-th percentile weight is selected as a threshold, with any weights with a value less than this being set to zero during the pruning process. It is possible to conduct pruning with this algorithm after training has occurred.
Relative threshold pruning can be utilised during training using the following:
python train_main.py --pruning_algorithm="threshold"
Low Magnitude Pruning
The default pruning algorithm utilised by the Tensorflow Model Optimisation Toolkit . In this implementation, the smallest 50% of weights are removed from the model during training.
TFMOT pruning can be utilised during training using the following:
python train_main.py --pruning_algorithm="tfmot"
Alternative Seq2Point Architectures
The default architecture used by Zhang et al. can be accessed using the following terminal command:
python train_main.py --network_type="default"
This architecture is used when no network type is provided upon initialising the training of a model.
Also available is a "reduced" architecture containing ten fewer filters per convolutional layer, and 2^9 instead of 2^10 hidden layer neurons. This can be accessed by changing the network type to
"reduced". The default architecture with dropout applied to each network layer can be utilised by replacing the network type with
"dropout", whilst the reduced
architecture with dropout applied can be used with
python train_main.py will train a seq2point model either with or without a pruning algorithm. The following options are available:
--appliance_name APPLIANCE_NAME The name of the appliance to train the network with. Default is kettle. Available are: kettle, fridge, washing machine, dishwasher, and microwave. --batch_size BATCH_SIZE The batch size to use when training the network. Default is 1000. --crop CROP The number of rows of the dataset to take training data from. Default is 10000. --pruning_algorithm PRUNING_ALGORITHM The pruning algorithm that the network will train with. Default is none. Available are: spp, entropic, threshold. --network_type NETWORK_TYPE The seq2point architecture to use. Only use if you do not want to use the standard architecture. Available are: standard, dropout, smaller, and optimal.
python test_main.py will test a pre-trained seq2point model. The following options are available:
--appliance_name APPLIANCE_NAME The name of the appliance to perform disaggregation with. Default is kettle. Available are: kettle, fridge, dishwasher, microwave. --batch_size BATCH_SIZE The batch size to use when training the network. Default is 1000. --crop CROP The number of rows of the dataset to take training data from. Default is 10000. --pruning_algorithm PRUNING ALGORITHM The pruning algorithm of the model to test. Default is none. --network_type NETWORK TYPE The seq2point architecture to use. Only use if you do not want to use the standard architecture. Available are: standard, dropout, smaller, and optimal.
 Zhang, C., Zhong., Wang, Z., Goddard, N., Sutton, S. (2017) Sequence-to_Point Learning with Neural Networks for Non-Intrusive Load Monitoring. Available from https://arxiv.org/pdf/1612.09106.pdf [accessed 17 January 2020].
 Wang, H., Zhang, Q., Wang, Y., Hu, H. (2018) Structured Probabilistic Pruning for Convolutional Neural Network Acceleration. Zhejiang University, China. Available from https://arxiv.org/pdf/1709.06994.pdf [accessed 17 January 2020].
 Hur, C., Kang, S. (2018) Entropy-Based Pruning Method For Convolutional Neural Networks. The Journal of Supercomputing, 75:2950–2963. Available from https://link-springer-com/content/pdf/10.1007/s11227-018-2684-z.pdf [accessed 17 January 2020].
 Tensorflow (undated). Magnitude-Based Weight Pruning with Keras. Available from https://www.tensorflow.org/model_optimization/guide/pruning/pruning_with_keras [accessed 17 January 2020].