Multi-view hybrid neural network for spatiotemporal semi-supervised sleep staging

LIU Hong-mei; PENG Cai-jing; HAN Fang; ZHANG Yuan

doi:10.13374/j.issn2095-9389.2022.03.22.005

Volume 45 Issue 5

May 2023

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Engineering > 2023 > 45(5): 797-806

LIU Hong-mei, PENG Cai-jing, HAN Fang, ZHANG Yuan. Multi-view hybrid neural network for spatiotemporal semi-supervised sleep staging[J]. Chinese Journal of Engineering, 2023, 45(5): 797-806. doi: 10.13374/j.issn2095-9389.2022.03.22.005

Citation:

LIU Hong-mei, PENG Cai-jing, HAN Fang, ZHANG Yuan. Multi-view hybrid neural network for spatiotemporal semi-supervised sleep staging[J]. Chinese Journal of Engineering, 2023, 45(5): 797-806. doi: 10.13374/j.issn2095-9389.2022.03.22.005

Citation:

PDF( 1175 KB)

Multi-view hybrid neural network for spatiotemporal semi-supervised sleep staging

doi: 10.13374/j.issn2095-9389.2022.03.22.005

1.
School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China
2.
Pediatric Respirology Department, The Ninth People’s Hospital of Chongqing, Chongqing 400700, China
3.
Sleep Medicine Center, Peking University People’s Hospital, Beijing 100044, China

More Information

Corresponding author: E-mail: yuanzhang@swu.edu.cn
Received Date: 2022-03-22
Available Online: 2022-08-10
Publish Date: 2023-05-01

Abstract

Abstract

Sleep takes approximately 1/3 of a person’s lifetime; therefore, its quality profoundly affects learning, physical recovery, and metabolism. Clinically relevant human physiological data are collected using polysomnography, which is analyzed by sleep technologists to determine sleep stages. However, the manual method is prone to having a cumbersome workload due to a large amount of data analysis and different data formats. Simultaneously, manually analyzed results are influenced by doctors’ medical clinical experience, which may cause inconsistent diagnoses. Recently, with the development of artificial intelligence, computer science, other technologies, and their interdisciplinarity, a series of typical achievements have been accomplished in intelligent diagnosis, laying the foundation for medical artificial intelligence in the sleep medicine field. In sleep research, realizing automatic sleep signal analysis and recognition assists doctors in diagnosis and reduces their workload, thus having important clinical significance and application value. Although deep neural networks are becoming popular for automatic sleep stage classification with supervised learning, large-scale, labeled datasets remain difficult to acquire. Learning from raw polysomnography signals and derived time-frequency image representations has been an interesting solution. However, extracting features from only a single dimension leads to inadequate feature extraction and, thus, limited accuracy. Hence, this paper aims to learn multi-view representations for physiological signals with semi-supervised learning. Specifically, we make the following contributions: (1) We propose a multi-view, hybrid neural network model containing a multichannel view time-frequency domain feature extraction mechanism, an attention mechanism, and a feature fusion module. Among these aspects, the multichannel view time-frequency domain mechanism extracts time domain and frequency domain signal features to achieve multi-view feature extraction. The attention mechanism module enhances salience features and achieves interclass feature extraction in the frequency domain. The feature fusion module fuses and classifies the above features. (2) A semi-supervised learning strategy is used to learn unlabeled electroencephalogram (EEG) data, which solves the problem of sleep data underutilization due to insufficient labeling of EEG signals in clinical practice. (3) Extensive experiments conducted on sleep stage classification demonstrate state-of-the-art performance compared with supervised learning and a semi-supervised baseline. Experimental results on three public databases (Sleep?EDF, DOD?H, and DOD?O) and one private database show that our semi-supervised method achieves accuracies of 81.6%, 81.5%, 79.2%, and 75.4%. The results show that our proposed model is comparable to a fully supervised sleep staging model while substantially reducing the technician’s workload in data labeling.
- sleep stage classification,
- semi-supervised,
- multi-view,
- hybrid neural network,
- electroencephalograph

FullText(HTML)

References(29)

References

[1]	Maquet P. The role of sleep in learning and memory. Science, 2001, 294(5544): 1048 doi: 10.1126/science.1062856
[2]	Berry R B, Brooks R, Gamaldo C, et al. AASM scoring manual updates for 2017 (version 2.4). J Clin Sleep Med, 2017, 13(5): 665 doi: 10.5664/jcsm.6576
[3]	張遠. 面向睡眠健康的感知、計算和干預. 科學通報, 2022, 67(1):27 Zhang Y. Sensing, computing and intervention for sleep health. Chin Sci Bull, 2022, 67(1): 27
[4]	Phan H, Andreotti F, Cooray N, et al. Joint classification and prediction CNN framework for automatic sleep stage classification. IEEE Trans Biomed Eng, 2019, 66(5): 1285 doi: 10.1109/TBME.2018.2872652
[5]	Chambon S, Galtier M N, Arnal P J, et al. A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Trans Neural Syst Rehabil Eng, 2018, 26(4): 758 doi: 10.1109/TNSRE.2018.2813138
[6]	Perslev M, Jensen M H, Darkner S, et al. U-time: A fully convolutional network for time series segmentation applied to sleep staging // Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, 2019: 1
[7]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition // IEEE Conference on Computer Vision and Pattern Recognition. New York, 2016: 770
[8]	Zhang H, Wu C, Zhang Z, et al. Resnest: Split-attention networks[J/OL]. arXiv preprint (2020-04-19) [2022-03-22].https://arxiv.org/abs/2004.08955
[9]	Phan H, Andreotti F, Cooray N, et al. Automatic sleep stage classification using single-channel EEG: Learning sequential features with attention-based recurrent neural networks // 40th Annual International Conference IEEE Engineering in Medicine and Biology Society, Honolulu, 2018: 1452
[10]	Phan H, Andreotti F, Cooray N, et al. SeqSleepNet: End-to-end hierarchical recurrent neural network for sequence-to- sequence automatic sleep staging. IEEE Trans Neural Syst Rehabil Eng, 2019, 27(3): 400 doi: 10.1109/TNSRE.2019.2896659
[11]	Phan H, Chén O Y, Koch P, et al. Towards more accurate automatic sleep staging via deep transfer learning. IEEE Trans Biomed Eng, 2021, 68(6): 1787 doi: 10.1109/TBME.2020.3020381
[12]	Supratak A, Dong H, Wu C, et al. DeepSleepNet: A model for automatic sleep stage scoring based on raw single-channel EEG. IEEE Trans Neural Sys Rehabil Eng, 2017, 25(11): 1998 doi: 10.1109/TNSRE.2017.2721116
[13]	Guillot A, Thorey V. Robust, et al. SleepNet: Transfer learning for automated sleep staging at scale. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 1441
[14]	Chen K, Zhang C, Ma J, et al. Sleep staging from single-channel EEG with multi-scale feature and contextual information. Sleep Breath, 2019, 23(4): 1159 doi: 10.1007/s11325-019-01789-4
[15]	Jia Z Y, Lin Y F, Wang J, et al. SalientSleepNet: Multimodal salient wave detection network for sleep staging // Thirtieth International Joint Conference on Artificial Intelligence. Montreal, 2021: 2614
[16]	金崢, 賈克斌, 袁野. 基于混合注意力時序網絡的睡眠分期算法研究. 生物醫學工程學雜志, 2021, 38(2):241 doi: 10.7507/1001-5515.202008006 Jin Z, Jia K B, Yuan Y. A hybrid attention temporal sequential network for sleep stage classification. J Biomed Eng, 2021, 38(2): 241 doi: 10.7507/1001-5515.202008006
[17]	Phan H, Chén O Y, Tran M C, et al. XSleepNet: Multi-view sequential model for automatic sleep staging. IEEE Trans Pattern Anal Mach Intell, 2022, 44(9): 5903
[18]	Fiorillo L, Favaro P, Faraci F D. DeepSleepNet-lite: A simplified automatic sleep stage scoring model with uncertainty estimates. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 2076 doi: 10.1109/TNSRE.2021.3117970
[19]	Dong H, Supratak A, Pan W, et al. Mixed neural network approach for temporal sleep stage classification. IEEE Trans Neural Syst Rehabil Eng, 2018, 26(2): 324 doi: 10.1109/TNSRE.2017.2733220
[20]	Tsinalis O, Matthews P M, Guo Y K, et al. Automatic sleep stage scoring with single-channel EEG using convolutional neural networks[J/OL]. arXiv preprint (2016-10-05) [2022-03-22]. https://arxiv.org/abs/1610.01683
[21]	Zhang H J, Wang J, Xiao Q F, et al. SleepPriorCL: Contrastive representation learning with prior knowledge-based positive mining and adaptive temperature for sleep staging[J/OL]. arXiv preprint (2021-10-15) [2022-3-22].https://arxiv.org/abs/2110.09966
[22]	Xiao Q, Wang J, Ye J, et al. Self-supervised learning for sleep stage classification with predictive and discriminative contrastive coding // 2021 IEEE International Conference on Acoustics Speech and Signal Processing. Toronto, 2021: 1290
[23]	Kemp B, Zwinderman A H, Tuk B, et al. Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG. IEEE Trans Biomed Eng, 2000, 47(9): 1185 doi: 10.1109/10.867928
[24]	Guillot A, Sauvet F, During E H, et al. Dreem open datasets: Multi-scored sleep datasets to compare human and automated sleep staging. IEEE Trans Neural Syst Rehabil Eng, 2020, 28(9): 1955 doi: 10.1109/TNSRE.2020.3011181
[25]	Sohn K, Berthelot D, Li C, et al. FixMatch: Simplifying semi-supervised learning with consistency and confidence. Adv Neural inf Process Syst, 2020, 33: 596
[26]	Korkalainen H, Aakko J, Nikkonen S, et al. Accurate deep learning-based sleep staging in a clinical population with suspected obstructive sleep apnea. IEEE J Biomed Health Inform, 2020, 24(7): 2073
[27]	Cui Y, Jia M L, Lin T Y, et al. Class-balanced loss based on effective number of samples // 2019 IEEE/CVF Conf Comput Vis Pattern Recognit. Long Beach, 2019: 9260
[28]	Zhang C H, Yu W W, Li Y M, et al. CMS2-net: Semi-supervised sleep staging for diverse obstructive sleep apnea severity. IEEE J Biomed Health Inform, 2022, 26(7): 3447 doi: 10.1109/JBHI.2022.3156585
[29]	Wu Z R, Xiong Y J, Yu S X, et al. Unsupervised feature learning via non-parametric instance discrimination // 2018 IEEE/CVF Conf Comput Vis Pattern Recognit. Salt Lake City, 2018: 3733