基于集成神經網絡的剩余壽命預測

摘要: 針對機器或設備的剩余壽命（Remaining useful life, RUL）預測精度低的問題，提出基于一維卷積神經網絡（Convolutional neural network, CNN）和雙向長短期記憶（Bidirectional long short-term memory, BD-LSTM）的集成神經網絡模型。為了更好地抽取時間序列上的特征，以及產生更多的訓練樣本，采用滑動窗口對數據進行處理，同時采用卡爾曼濾波對數據進行降噪處理，將數據標準化以及設置RUL標簽。與人工提取特征不同，利用一維CNN對數據進行特征提取，并舍棄了CNN中的池化層。然后將提取到的高維特征輸入到BD-LSTM進行回歸預測，并采用Bagging的方式對此神經網絡進行集成來預測RUL。最后通過在NASA的數據集上驗證該模型的有效性，以及相比于其他機器學習或者深度學習模型的優越性，實驗表明所提模型在RUL預測方面更加準確。
- 卡爾曼濾波 /
- 剩余壽命預測 /
- 神經網絡 /
- 深度學習 /
- 集成學習
Abstract: Unexpected failures and unscheduled maintenance activities of mechanical systems might incur considerable waste of resources and high investment costs. Thus, in recent years, prognostics and health management (PHM) has received a lot of attention because of its importance in maintenance cost reduction and machine fault prognostics. The remaining useful life (RUL) of machinery is defined as the length from the current time to the end of its useful life, which is the core technology of PHM. During the operation of machines and equipment, a large amount of data generated by different sensors in the system is collected using various methods. These data often characterize the health status of machinery to a certain extent. By applying the systematic approach to these data, valuable information for strategic decision-making can be obtained. However, traditional machine learning algorithms are usually not efficient enough to handle the complex and nonlinear characteristics of the system and deal with big data. With the rapid development of modern computational hardware and theory, deep learning algorithms show unique advantages in characterizing the system complexity and processing big data. Because of the low-accuracy prediction of the RUL of machines or equipment, a neural network integrating the one-dimensional convolutional neural network (1D CNN) and the bidirectional long short-term memory (BD-LSTM) was proposed. To extract the features of the time series and generate more training samples, the sliding window algorithm was used to process the data and the Kalman filter was applied to denoise the data. Then, the dataset was standardized and the RUL labels were set. Instead of artificial feature extraction, this study used 1D CNN to extract features from the data and discarded the pooling layer of CNN. The extracted high-dimensional features were inputted into the BD-LSTM for regression prediction, and the neural network was integrated by bagging to predict the RUL. Finally, the effectiveness and superiority of the model compared with the machine or deep learning model were verified using the National Aeronautics and Space Administration dataset. Results showed that the proposed model can more accurately predict the RUL than the machine or deep learning model.
- Kalman filter /
- remaining useful life prediction /
- neural network /
- deep learning /
- ensemble learning

HTML全文

圖 1 一維CNN的操作示意圖

Figure 1. Illustration of the one-dimensional convolutional neural network operation

下載: 全尺寸圖片幻燈片

圖 2 LSTM單元結構示意圖

Figure 2. Diagram of the LSTM cell

下載: 全尺寸圖片幻燈片

圖 3 雙向LSTM操作示意圖

Figure 3. Diagram of the bidirectional LSTM network

下載: 全尺寸圖片幻燈片

圖 4 模型框架

Figure 4. Model framework

下載: 全尺寸圖片幻燈片

圖 5 不同的RUL標簽對比

Figure 5. Comparison of different RUL labels

下載: 全尺寸圖片幻燈片

圖 6 預處理前后的傳感器數據。（a,c）s12傳感器；（b,d） s2傳感器

Figure 6. Sensor data before and after preprocessing: (a,c) Sensor 12; (b,d) Sensor 2

下載: 全尺寸圖片幻燈片

圖 7 卷積層權值分布。（a）第1個卷積層；（b）第2個卷積層；（c）第3個卷積層；（d）第4個卷積層

Figure 7. Convolutional layer weight distribution: (a) the first convolutional layer; (b) the second convolutional layer; (c) the third convolutional layer; (d) the fourth convolutional layer

下載: 全尺寸圖片幻燈片

圖 8 卷積層數目對RMSE的影響

Figure 8. Effect of the number of convolution layers on the root–mean–square error

下載: 全尺寸圖片幻燈片

圖 9 不同評價函數的對比

Figure 9. Comparison of different evaluation functions

下載: 全尺寸圖片幻燈片

圖 10 訓練過程的loss變化

Figure 10. Loss changes during the training process

下載: 全尺寸圖片幻燈片

圖 11 神經元個數對評價指標的影響

Figure 11. Influence of the number of neurons on the evaluation metric

下載: 全尺寸圖片幻燈片

圖 12 時間窗口大小對評價指標的影響

Figure 12. Influence of the time window size on the evaluation metric

下載: 全尺寸圖片幻燈片

圖 13 測試集真實RUL與預測RUL的對比

Figure 13. Comparison of real and predicted RUL in the test set

下載: 全尺寸圖片幻燈片

表 1 基學習器網絡層次表

Table 1. Network hierarchy table of the base learner

Network structure	Input shape	Output shape
Conv1D	（30,14）	（21,8）
Conv1D	（21,8）	（12,16）
Conv1D	（12,16）	（10,32）
BD-LSTM	（10,32）	（256）
Dropout	（256）	（256）
Output	（256）	（1）

下載: 導出CSV

表 2 各種方法結果的對比

Table 2. Comparison of the results of various methods

No.	Method	RMSE	Score
1	MLP^[17]	37.56	17972
2	SVR^[17]	20.96	1381
3	CNN^[17]	18.44	1286
4	RVR^[17]	23.80	1500
5	LSTM^[24]	16.73	388
6	KNR^[25]	20.46	729
7	RF^[25]	17.91	479
8	BD-RNN^[26]	18.07	N/A
9	CNN+BD-LSTM	15.10	344
10	Integrated CNN+BD-LSTM	14.47	311

下載: 導出CSV

259luxu-164

參考文獻(26)

[1]	Uckun S, Goebel K, Lucas P J F. Standardizing research methods for prognostics // 2008 International Conference on Prognostics and Health Management. Denver, 2008: 1
[2]	Tang D Y, Makis V, Jafari L, et al. Optimal maintenance policy and residual life estimation for a slowly degrading system subject to condition monitoring. Reliab Eng Syst Saf, 2015, 134: 198 doi: 10.1016/j.ress.2014.10.015
[3]	Canizo M, Onieva E, Conde A, et al. Real-time predictive maintenance for wind turbines using Big Data frameworks // 2017 IEEE International Conference on Prognostics and Health Management (ICPHM). Dallas, 2017: 70
[4]	Lei Y G, Li N P, Gontarz S, et al. A model-based method for remaining useful life prediction of machinery. IEEE Trans Reliab, 2016, 65(3): 1314 doi: 10.1109/TR.2016.2570568
[5]	Si X S, Wang W B, Hu C H, et al. Remaining useful life estimation – A review on the statistical data driven approaches. Eur J Oper Res, 2011, 213(1): 1 doi: 10.1016/j.ejor.2010.11.018
[6]	Liu Y C, Hu X F, Zhang W J. Remaining useful life prediction based on health index similarity. Reliab Eng Syst Saf, 2019, 185: 502 doi: 10.1016/j.ress.2019.02.002
[7]	Long Y W, Luo H W, Zhi Y, et al. Remaining useful life estimation of solder joints using an ARMA model optimized by genetic algorithm // 2018 19th International Conference on Electronic Packaging Technology (ICEPT). Shanghai, 2018: 1108
[8]	Wu W, Hu J T, Zhang J L. Prognostics of machine health condition using an improved ARIMA-based prediction method // 2007 2nd IEEE Conference on Industrial Electronics and Applications. Harbin, 2007: 1062
[9]	Zhou Y P, Huang M H. Lithium-ion batteries remaining useful life prediction based on a mixture of empirical mode decomposition and ARIMA model. Microelectron Reliab, 2016, 65: 265 doi: 10.1016/j.microrel.2016.07.151
[10]	Tian Z G. An artificial neural network method for remaining useful life prediction of equipment subject to condition monitoring. J Intell Manuf, 2012, 23(2): 227 doi: 10.1007/s10845-009-0356-9
[11]	Mosallam A, Medjaher K, Zerhouni N. Data-driven prognostic method based on Bayesian approaches for direct remaining useful life prediction. J Intell Manuf, 2016, 27(5): 1037 doi: 10.1007/s10845-014-0933-4
[12]	Khelif R, Chebel-Morello B, Malinowski S, et al. Direct remaining useful life estimation based on support vector regression. IEEE Trans Ind Electron, 2017, 64(3): 2276 doi: 10.1109/TIE.2016.2623260
[13]	Miao Q, Xie L, Cui H J, et al. Remaining useful life prediction of lithium-ion battery with unscented particle filter technique. Microelectron Reliab, 2013, 53(6): 805 doi: 10.1016/j.microrel.2012.12.004
[14]	Tobon-Mejia D A, Medjaher K, Zerhouni N, et al. A data-driven failure prognostics method based on mixture of Gaussians hidden Markov models. IEEE Trans Reliab, 2012, 61(2): 491 doi: 10.1109/TR.2012.2194177
[15]	Li Z X, Wu D Z, Hu C, et al. An ensemble learning-based prognostic approach with degradation-dependent weights for remaining useful life prediction. Reliab Eng Syst Saf, 2019, 184: 110 doi: 10.1016/j.ress.2017.12.016
[16]	Heimes F O. Recurrent neural networks for remaining useful life estimation // 2008 International Conference on Prognostics and Health Management. Denver, 2008: 1
[17]	Babu G S, Zhao P L, Li X L. Deep convolutional neural network based regression approach for estimation of remaining useful life // International Conference on Database Systems for Advanced Applications. Dallas, 2016: 214
[18]	Yuan M, Wu Y T, Lin L. Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network // 2016 IEEE International Conference on Aircraft Utility Systems (AUS). Beijing, 2016: 135
[19]	Zhang Y Z, Xiong R, He H W, et al. Long short-term memory recurrent neural network for remaining useful life prediction of lithium-ion batteries. IEEE Trans Veh Technol, 2018, 67(7): 5695 doi: 10.1109/TVT.2018.2805189
[20]	Ordó?ez C, Lasheras F S, Roca-Pardi?as J, et al. A hybrid ARIMA–SVM model for the study of the remaining useful life of aircraft engines. J Comput Appl Math, 2019, 346: 184 doi: 10.1016/j.cam.2018.07.008
[21]	Guo L, Li N P, Jia F, et al. A recurrent neural network based health indicator for remaining useful life prediction of bearings. Neurocomputing, 2017, 240: 98 doi: 10.1016/j.neucom.2017.02.045
[22]	Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput, 1997, 9(8): 1735 doi: 10.1162/neco.1997.9.8.1735
[23]	Wu Y T, Yuan M, Dong S P, et al. Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 2018, 275: 167 doi: 10.1016/j.neucom.2017.05.063
[24]	Zheng S, Ristovski K, Farahat A, et al. Long short-term memory network for remaining useful life estimation // 2017 IEEE International Conference on Prognostics and Health Management (ICPHM). Dallas, 2017: 88
[25]	Zhang C, Lim P, Qin A K, et al. Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE Trans Neural Networks Learn Syst, 2017, 28(10): 2306 doi: 10.1109/TNNLS.2016.2582798
[26]	Zhang J J, Wang P, Yan R Q, et al. Long short-term memory for machine remaining life prediction. J Manuf Syst, 2018, 48: 78 doi: 10.1016/j.jmsy.2018.05.011