Paper: Anomaly Detection of Wind Turbine Time Series using Variational Recurrent Autoencoders.
Ice accumulation in the blades of wind turbines can cause them to describe anomalous rotations or no rotations at all, thus affecting the generation of electricity and power output. In this work, we investigate the problem of ice accumulation in wind turbines by framing it as anomaly detection of multi-variate time series. Our approach focuses on two main parts: first, learning low-dimensional representations of time series using a Variational Recurrent Autoencoder (VRAE), and second, using unsupervised clustering algorithms to classify the learned representations as normal (no ice accumulated) or abnormal (ice accumulated). We have evaluated our approach on a custom wind turbine time series dataset, for the two-classes problem (one normal versus one abnormal class), we obtained a classification accuracy of up to 96% on test data. For the multiple-class problem (one normal versus multiple abnormal classes), we present a qualitative analysis of the low-dimensional learned latent space, providing insights into the capacities of our approach to tackle such problem.
-
Install Anaconda and create an environment in your system.
-
Install the following dependencies in your anaconda environment
- NumPy (>= 1.19.2)
- Matplotlib (>= 3.3.2)
- PyTorch (>= 1.7.0)
- scikit-learn (>= 0.23.2)
- tqdm
- time
- ast
- pathlib
- pickle
Our dataset is composed of wind turbine multivariate time series simulated with an open source whole-turbine simulator called FAST. We model a wind turbine operating with and without ice on the rotor blades. The rotor blades are divided in three zones: one covers the first half of the blade and the other two zones divide the second half into two extra halves again. For each simulation, each one of these three zones contain a particular ice mass. The convention to refer to the region and amount of ice mass is x − y − z. For example, a configuration 0.4 − 0.6 − 0.8 implies 0.4 kg of ice in zone one, 0.6 kg in zone two and 0.8 kg in zone three.
We have balanced the dataset to have approximately the same number of normal and abnormal time series. It is composed of 14 normal and 11 abnormal simulations. Each simulation corresponds to a time series consisting of 10,000 time steps and 27 sensor features in total. First, we minmax scale the data to the range [-1 to 1]. This is a crucial preprocessing step since the learning performance of the model can be affected by the various sensor amplitudes recorded. Second, we select only 6 out of the the initial 27 features. These features are the accelerations in flapwise and edgewise components for three blades. We have chosen these features based on the idea that this physical information is sufficient for the model to be able to cluster time series in lower dimensions efficiently.
Next, we reshape the data to the format (samples, timesteps, features), this is the required input format for LSTM in pytorch. Finally, we slice each time series into smaller segments of 200 to 1000 time steps. The simulations are set up such that we have roughly 12 rotations per minute, this means there is one rotation every 5 seconds, this corresponds to a length of approximately 1000 time steps. This procedure generates a total of 1250 chunks (normal and abnormal) that are split into 70% training and 30% validation sets.
The figure below shows the implemented architecture: Variational Recurrent Autoencoder (VRAE) that uses an LSTM as a hidden layer.
This is an example code of how the encoder and decoder are defined in pytorch:
class Encoder(nn.Module):
"""
"""
def __init__(self, number_of_features, hidden_size_1, hidden_layer_depth,
latent_length, dropout, block = 'LSTM'):
super(Encoder, self).__init__()
self.number_of_features = number_of_features
self.hidden_size_1 = hidden_size_1
self.hidden_layer_depth = hidden_layer_depth
self.latent_length = latent_length
if block == 'LSTM':
self.model = nn.LSTM(self.number_of_features, self.hidden_size_1,
self.hidden_layer_depth, dropout = dropout,
bidirectional = False) # checking bi-directional mode
print("Single LSTM model")
print(self.model)
print(type(self.model))
elif block == 'GRU':
self.model = nn.GRU(self.number_of_features, self.hidden_size_1,
self.hidden_layer_depth, dropout = dropout)
else:
raise NotImplementedError
class Decoder(nn.Module):
"""
"""
def __init__(self, sequence_length, batch_size, hidden_size_1,
hidden_layer_depth, latent_length, output_size,
dtype, block='LSTM'):
super(Decoder, self).__init__()
self.hidden_size_1 = hidden_size_1
self.batch_size = batch_size
self.sequence_length = sequence_length
self.hidden_layer_depth = hidden_layer_depth
self.latent_length = latent_length
self.output_size = output_size
self.dtype = dtype
if block == 'LSTM':
self.model = nn.LSTM(1, self.hidden_size_1, self.hidden_layer_depth)
elif block == 'GRU':
self.model = nn.GRU(1, self.hidden_size_1, self.hidden_layer_depth)
else:
raise NotImplementedError
self.latent_to_hidden = nn.Linear(self.latent_length, self.hidden_size_1)
self.hidden_to_output = nn.Linear(self.hidden_size_1, self.output_size)
self.decoder_inputs = \
torch.zeros(self.sequence_length, self.batch_size, 1,
requires_grad=True).type(self.dtype)
self.c_0 = \
torch.zeros(self.hidden_layer_depth, self.batch_size,
self.hidden_size_1, requires_grad=True).type(self.dtype)
# Xavier weight intilization
nn.init.xavier_uniform_(self.latent_to_hidden.weight)
nn.init.xavier_uniform_(self.hidden_to_output.weight)The main.py script calls the VRAE model and trains it on the wind turbine data. It takes 1 hour to train for 500 epochs using an NVIDIA Tesla V100 GPU.
To run this model, open a terminal, activate your conda environment and type
python main.py -df 1 --hidden_size_1 128 --latent_length 5 -bs 64 -ne 100
The following are some hyperparameters that can be modified to train the model
--hidden_size_1the number of features in the hidden state h--latent_lengthdimension of latent vectors--batch_sizebatch size for training--n_epochsnumber of epochs
After training the model, a file with the model parameters is generated. Also, a set of images that project latent vectors into 2D are generated automatically.
- Improve clustering on multiple classes case (there is an overlap between zone 2 and zone 3 as shown above)
- Train model on new dataset (different simulation conditions)
- Improve time series reconstructions