深度残差收缩网络

residual

2019/09/27 发布于 研究 分类

一种深度学习方法

深度残差学习  深度学习  深度神经网络 

文字内容
1. Deep Residual Shrinkage Networks for Fault Diagnosis Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, and Michael Pecht M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 1
2. Backgrounds  Accurate fault diagnosis of mechanical systems is useful to schedule maintenance and extend service time as well as ensure human safety.  In recent years, deep learning methods, which refer to the artificial intelligence methods with multiple levels of nonlinear transformations, have become a useful tool in vibration-based fault diagnosis.  Deep residual networks (ResNets) are an attractive deep learning method, which use identity shortcuts to ease the difficulty of parameter optimization. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 2
3. Motivations  Vibration signals of large rotating machineries, such as wind turbines, manufacturing machines, and heavy trucks, contain large amounts of noise.  The feature learning ability of deep learning methods often decreases when dealing with highly noised vibration signals.  This paper proposes two deep residual shrinkage networks (DRSNs), i.e., a DRSN with channel-shared thresholds (DRSN-CS) and a DRSN with channel-wise thresholds (DRSN-CW), to improve the feature learning ability of ResNets from highly noised vibration signals, with the ultimate objective of achieving high diagnostic accuracy. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 3
4. Innovations  Soft thresholding (i.e., a popular shrinkage function) is inserted into the deep architecture as nonlinear transformation layers, in order to effectively eliminate the noise-related features.  The thresholds are adaptively determined using specially designed subnetworks so that each vibration signal can have its own set of thresholds.  Two kinds of thresholds, namely, channel-shared thresholds and channel-wise thresholds, are considered in soft thresholding, which is the reason for the terms DRSN-CS and DRSN-CW. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 4
5. Basics of ResNets Three kinds of RBUs, including (a) an RBU in which the output feature map is the same size as the input feature map, (b) an RBU with a stride of 2, in which the width of the output feature map is reduced to half of that of the input feature map, and (c) an RBU with a stride of 2 and a doubled number of convolutional kernels, in which the number of channels of the output feature map is doubled. (d) The overall architecture of a ResNet. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 5
6. Basics of soft thresholding (a) Soft thresholding and (b) its derivative. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 6
7. The developed DRSN-CS (a) A building unit entitled residual shrinkage building unit with channel-shared thresholds (RSBU-CS), (b) an overall architecture of DRSN-CS M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 7
8. The developed DRSN-CW (c) A building unit entitled residual shrinkage building unit with channel-wise thresholds (RSBU-CW), (d) an overall architecture of DRSN-CW M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 8
9. Results (a) Training and (b) test accuracies of ConvNet, ResNet, DRSN-CS, and DRSNCW for fault diagnosis with different amounts of Gaussian noise. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 9
10. Results (a) Training and (b) test accuracies of ConvNet, ResNet, DRSN-CS, and DRSNCW for fault diagnosis with different amounts of Laplacian noise. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 10
11. Results (a) Training and (b) test accuracies of ConvNet, ResNet, DRSN-CS, and DRSNCW for fault diagnosis with different amounts of pink noise. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 11
12. Results 2D visualizations of high-dimensional features at the final GAP layer of testing observations in (a) the ConvNet, (b) the ResNet, (c) the DRSN-CS, and (d) the DRSN-CW, when SNR = 5 dB with Gaussian noise. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 12
13. Results Training and test errors obtained from (a) the ConvNet, (b) the ResNet, (c) the DRSN-CS, and (d) the DRSN-CW, when SNR = 5dB with Gaussian noise. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 13
14. Conclusions  The integration of soft thresholding as trainable shrinkage functions in deep learning methods can effectively improve the discriminative feature learning ability from highly noised vibration signals.  The developed DRSNs are not only applicable to fault diagnosis tasks using vibration signals, but also to pattern recognition tasks in a variety of fields when dealing with various kinds of signals that interfere with noise, such as acoustic signals, visual signals, and current signals. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 14
15. References 1. M. Zhao, M. Kang, B. Tang, and M. Pecht, “Deep Residual Networks With Dynamically Weighted Wavelet Coefficients for Fault Diagnosis of Planetary Gearboxes,” IEEE Transactions on Industrial Electronics, vol. 65, no. 5, pp. 4290–4300, 2018. 2. M. Zhao, M. Kang, B. Tang, and M. Pecht, "Multiple Wavelet Coefficients Fusion in Deep Residual Networks for Fault Diagnosis," IEEE Transactions on Industrial Electronics, vol. 66, no. 6, pp. 4696-4706, 2019. 3. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. CVPR, Seattle, WA, USA, Jun. 27–30, 2016, pp. 770–778. 4. K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in Computer Vision—ECCV 2016 (Lecture Notes in Computer Science 9908), B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds., Cham, Switzerland: Springer, 2016, pp. 630–645. 5. D. L. Donoho, “De-noising by soft-thresholding,” IEEE Transactions on Information Theory, vol. 41, no. 3, pp. 613–627, 1995. 6. J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proc. IEEE Conf. CVPR, Salt Lake City, UT, USA, Jun. 18–23, 2018, pp. 7132–7141. 7. K. Isogawa, T. Ida, T. Shiodera, and T. Takeguchi, “Deep shrinkage convolutional neural network for adaptive noise reduction,” IEEE Signal Processing Letters, vol. 25, no. 2, pp. 224–228, 2018. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 15