NASA Turbofan Engine Fault Dataset @ AWS# Predictive Maintenance using Machine Learning on Sagemaker 3 Day AWS workshop and 2 day ML project class work.
S3 : data location : https://sagemaker-us-east-1-023375022819.s3.amazonaws.com/PredictiveMaintenance-Team-9/input/CMAPSSData.zip
Imagine you are the manager at a manufacturing company responsible for monitoring assembly lines. Each assembly line contains multiple kinds of machines that must work continuously and reliably to produce ready-to-ship products as can be seen in the image below. IoT sensors placed on these machines monitor electricity consumption, noise generated, vibration, temperature and various other measurable quantities that are used to monitor the health of each machine. Sudden breakdown of any of these machines across multiple assembly lines will lead to:
- Unscheduled downtime and resulting delays in delivering your product to market
- Cost incurred due to delays, and hiring maintenance workers to repair and possibly replace parts of the machine that caused the breakdown
You have been tasked with researching a technique called “predictive maintenance”, especially after your competitors Advantech, Inc. have published a report (http://www.advantech.com/industrial-automation/industry4.0/pms#my_cen). Additionally, you are intrigued to see if Machine Learning can help with this problem. Your team's collective research notes regarding a potential proof-of-concept that you will be building is included here:
Create ML model that can predict Remaining Useful Life (RUL) on an engine after each run cycle to improve the Predictive Maintenance process. With a predictive model, the Maintenance Department can use the information to determine when to perform preventative maintenance on an engine. This timing improvement will minimize engine down time resulting in increased production output and lower costs.
Turbofan Engine Degradation Simulation Data Set Publications using this data set Description Engine degradation simulation was carried out using C-MAPSS. Four different were sets simulated under different combinations of operational conditions and fault modes. Records several sensor channels to characterize fault evolution. The data set was provided by the Prognostics CoE at NASA Ames. Format The set is in text format and has been zipped including a readme file. Datasets + Download Turbofan Engine Degradation Simulation Data Set (39327 downloads) Dataset Citation A. Saxena and K. Goebel (2008). "Turbofan Engine Degradation Simulation Data Set", NASA Ames Prognostics Data Repository (http://ti.arc.nasa.gov/project/prognostic-data-repository), NASA Ames Research Center, Moffett Field, CA
colaboration spot in sage maker: https://bhavanashah-aws-ml.notebook.us-east-1.sagemaker.aws/tree/PredictiveMaintenance
More about the xGboosst model used : https://aws.amazon.com/blogs/machine-learning/simplify-machine-learning-with-xgboost-and-amazon-sagemaker/
Once main train dataset was created, we split it further into train/validation by 80/20. We used XGBoost regression algorithm, that is a supervised learning algorithm that attempts to accurately predict a target variable by combining an ensemble of estimates from a set of simpler, weaker models First we trained model with static hyperparameters Then reconfigured for Hyperparameter tuning
"max_depth":"5", "eta":"0.2", "gamma":"4", "min_child_weight":"6", "subsample":"0.7", "silent":"0", "objective":"reg:linear", "num_round":"50”
'eta': ContinuousParameter(0.3, 0.7), 'min_child_weight': ContinuousParameter(5, 10), 'alpha': ContinuousParameter(0, 2), 'max_depth': IntegerParameter(4, 8)
Maintenance schedules in typical manufacturing and energy companies that involve large number of machines performing tasks are typically a result of “reactive” or “preventive” maintenance. A reactive maintenance task is scheduled if a machine breaks down (or fails), or is operating in a known degrade state of operation. Preventive maintenance is triggered by usage or time or a fixed schedule. As an example, for car owners, reactive maintenance occurs after there is an failure of a component (stalled engine, punctured tire, etc.), whereas preventive maintenance occurs on a fixed schedule (for example, tire rotation or oil change every 10000 miles) even though there may not be a need for doing so.
“Predictive” maintenance implies that maintenance is scheduled when a system predicts the possible occurrence of a failure even in the future. This solves problems that are common in reactive and preventive maintenance - 1. reactive maintenance adds unnecessary time and cost to project schedules, since maintenance workers are deployed only after discovery of a failure event; and 2. preventive maintenance adds unnecessarily frequent maintenance tasks, therefore increasing wait times and costs for the end user. Currently, predictive maintenance techniques involve simple monitoring-and-thresholding, or statistical techniques to identify anomalies from sensor data. However, these techniques are limited to use by Subject Matter Experts (SMEs) and depend on human-generated thresholds. With Machine Learning (ML), it is possible to train models to detect abnormal patterns from sensor data. The trained ML model does not require rules or pre-programmed thresholds, and vast amounts of data can be analyzed repeatably with no need of human involvement.
Background
NASA's Prognostic Center of Excellence established a repository with datasets to be used for benchmarking prognostics and predictive maintenance related algorithms. Among these datasets involves data from a turbofan engine simulation model C-MAPPS (or Commercial Modular Aero Propulsion System Simulation). The references section contains details about the over 100 publications using this dataset. C-MAPPS is a tool used to generate health, control and engine parameters from a simulated turbofan engine. A custom code wrapper was used to inject synthetic faults and continuous degradation trends into a time series of sensor data. Some high level characteristics of this dataset are as follows:
- The data obtained is from a high fidelity simulation of a turbofan engine, but closely models the sensor values of an actual engine.
- Synthetic noise was added to the dataset to replicate real-world scenarios.
- The effects of faults are masked due to operational conditions, which is a common trait of most real world systems.
Download the dataset here - https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/#turbofan.
Data sets consists of multiple multivariate time series. Each data set is further divided into training and test subsets. Each time series is from a different engine ñ i.e., the data can be considered to be from a fleet of engines of the same type. Each engine starts with different degrees of initial wear and manufacturing variation which is unknown to the user. This wear and variation is considered normal, i.e., it is not considered a fault condition. There are three operational settings that have a substantial effect on engine performance. These settings are also included in the data. The data is contaminated with sensor noise.
The engine is operating normally at the start of each time series, and develops a fault at some point during the series. In the training set, the fault grows in magnitude until system failure. In the test set, the time series ends some time prior to system failure. The objective of the competition is to predict the number of remaining operational cycles before failure in the test set, i.e., the number of operational cycles after the last cycle that the engine will continue to operate. Also provided a vector of true Remaining Useful Life (RUL) values for the test data.
The data are provided as a zip-compressed text file with 26 columns of numbers, separated by spaces. Each row is a snapshot of data taken during a single operational cycle, each column is a different variable. The columns correspond to:
- unit number
- time, in cycles
- operational setting 1
- operational setting 2
- operational setting 3
- sensor measurement 1
- sensor measurement 2 ...
- sensor measurement 26
Source: https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/#turbofan
Downloaded locally into the SageMaker Instance and to the S3 bucket s3://sagemaker-us-east-1-023375022819/PredictiveMaintenance-Team-9/input/CMAPSSData
Data Wrangling in Jupyter Notebook
- For each type of dataset in train and test: FD001, FD002, FD003, FD004 Create a new variables for Conditions: 1 - ONE (Sea Level) 2 – SIX Fault Mode: 1 – ONE (HPC Degradation) or 2 – TWO (HPC Degradation, Fan Degradation)
- Create RUL target label in train dataset, which is calculated as finding the max cycle for that engine, assigning that as output value for first RUL corresponding to first cycle and decrementing that by one within that engine
- Create RUL target label in test dataset, which is calculated as finding the max cycle for that engine, from the RUL_FD00~.txt file, assigning that as output value for first RUL corresponding to first cycle and decrementing that by one within that engine
- Remove unnecessary columns
- Concatenate all train files and test files respectively
- Perform One-Hot Encoding, to create columns condition-1, condition-2, faultMode-1, faultMode-2
We strongly recommend thinking about this as a regression problem. Here's an example notebook to get you started with regression using XGBoost.
Predictive maintenance using the turbofan engine dataset has been thought of as one of the following ML problems:
- Classification problem -
- predict if the engine will fail in a particular time window (yes / no)
- predict which windows the engine will fail in (e.g., window 0, window 3, and window 4)
- Regression problem -
- predict Time to Failure (TTF)
- predict Remaining Useful Life (RUL)
The data provided is not in a format suitable for all of the above ways of thinking about the predictive maintenance ML problem. How will you modify the data to suit your model?
Are all features important? Can some features be combined or aggregated?
Are your classes (if you have any) evenly distributed? If not, how do you help balance this class distribution?
How do you evaluate the accuracy of your model? What cells in the confusion matrix (if you have one) have higher impact for a fictitious turbofan engine manufacturer that uses your model?
We used a naive assumption, that the degradation starts at the very first cycle of operation. Deeper understanding and research will help to find the optimal threshold value for when actual degradation could have started to get a more accurate Remaining Useful Life (RUL).
References
- https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/publications/#turbofan
- https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20150007677.pdf
- A. Saxena, K. Goebel, D. Simon, and N. Eklund, “Damage Propagation Modeling for Aircraft Engine Run-to-Failure Simulation”, in the Proceedings of the Ist International Conference on Prognostics and Health Management (PHM08), Denver CO, Oct 2008.