上海交通大学学报（医学版）

上海交通大学学报（医学版） >

2022 , Vol. 42 >Issue 5: 653 - 659

DOI: https://doi.org/10.3969/j.issn.1674-8115.2022.05.014

论著 · 技术与方法

基于深度卷积神经网络的Barrett食管内镜图片分类模型的建立

  • 林嘉希 ,
  • 汪盛嘉 ,
  • 赵鑫 ,
  • 高欣 ,
  • 殷民月 ,
  • 朱锦舟
展开
  • 1.苏州大学附属第一医院消化内科，苏州 215006
    2.江苏省苏州市消化病临床医学中心，苏州 215006
    3.苏州大学附属第一医院普外科，苏州 215006
林嘉希（1999—），男，本科生；电子信箱：jxlin@stu.suda.edu.cn。
朱锦舟，电子信箱: jzzhu@zju.edu.cn。

收稿日期: 2022-01-07

  录用日期: 2022-03-25

  网络出版日期: 2022-05-07

基金资助

国家自然科学基金(82000540);苏州市“科教兴卫”青年科技项目(KJXW2019001)

收起

Development of endoscopic image classification models of Barrett's esophagus based on deep convolutional neural networks

  • Jiaxi LIN ,
  • Shengjia WANG ,
  • Xin ZHAO ,
  • Xin GAO ,
  • Minyue YIN ,
  • Jinzhou ZHU
Expand
  • 1.Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou 215006, China
    2.Suzhou Clinical Center of Digestive Disease, Jiangsu Province, Suzhou 215006, China
    3.Department of General Surgery, The First Affiliated Hospital of Soochow University, Suzhou 215006, China
ZHU Jinzhou, E-mail: jzzhu@zju.edu.cn.

Received date: 2022-01-07

  Accepted date: 2022-03-25

  Online published: 2022-05-07

Supported by

National Natural Science Foundation of China(82000540);Youth Program of Suzhou Health Committee(KJXW2019001)

Fold

摘要

目的·利用深度卷积神经网络算法，构建Barrett食管内镜图片分类模型并评估其分类能力。方法·收集苏州大学附属第一医院消化内镜中心及HyperKvasir数据库中的内镜下食管图片共806张，其中正常食管图片412张、Barrett食管图片394张。随机将所有图片分为训练集（85%）与验证集（15%）。利用于ImageNet数据库进行预训练的4种深度卷积神经网络［Xception、NASNet Large（NASNetL）、ResNet50V2（ResNet）及BigTransfer（BiT）］，分别在训练集中进行迁移学习，建立Barrett食管内镜图片分类模型，并使用梯度加权分类激活映射对该4个模型的分类结果进行可视化解释。随后，于验证集中评价模型的分类能力。同时，收集高、低年资医师对验证集数据的分类结果，将其与模型的分类结果进行对比，进一步评估模型的分类能力。结果·成功构建了基于深度卷积神经网络的Barrett食管内镜图片的4个分类模型。利用梯度加权分类激活映射，以热力图形式实现了对模型分类结果的可视化解释。在验证集数据中，各模型均拥有较高的分类准确性与精确性，其平均分类准确性为0.852，平均分类精确性为0.846。NASNetL模型相较其余3种模型，拥有最高分类准确性（0.873）及最高分类精确性（0.867），是表现最优的模型。该模型对Barrett食管内镜图片的分类能力近似高年资医师，其分类准确性略低于高年资医师（0.881）而高于低年资医师（0.855）；同时，该模型与高年资医师（Kappa=0.712，P=0.000）、低年资医师（Kappa=0.695，P=0.000）均具有较好的分类一致性。结论·利用深度卷积神经网络迁移学习构建的Barrett食管内镜图片分类模型具有较好的分类能力。

关键词： Barrett食管; 深度学习; 迁移学习; 消化内镜; 卷积神经网络

本文引用格式

林嘉希 , 汪盛嘉 , 赵鑫 , 高欣 , 殷民月 , 朱锦舟 . 基于深度卷积神经网络的Barrett食管内镜图片分类模型的建立[J]. 上海交通大学学报（医学版）, 2022 , 42(5) : 653 -659 . DOI: 10.3969/j.issn.1674-8115.2022.05.014

Abstract

Objective

·To develop classification models for endoscopic images of Barrett's esophagus using deep convolutional neural network, and evaluate its classification performance.

Methods

·A total of 806 esophageal images were collected from Gastrointestinal Endoscopy Center of the First Affiliated Hospital of Soochow University and the HyperKvasir database, including 412 normal esophageal images and 394 Barrett's esophagus images. All the images were randomly divided into training set (85%) and validation set (15%). Four deep convolutional neural networks [Xception, NASNet Large (NASNetL), ResNet50V2 (ResNet) and BigTransfer (BiT)], which had been pre-trained in the ImageNet database, were used to perform transfer learning in the training set and develop endoscopic image classification models of Barrett's esophagus, respectively. The gradient-weighted class activation mapping (Grad-CAM) was used to visually interpret the classification results of the four models. In the validation set, the models' classification abilities were evaluated. Besides, to further evaluate the classification abilities of the models, the classification results of the validation set by senior and junior physicians were compared with the models'.

Results

·The 4 endoscopic image classification models of Barrett's esophagus based on deep convolutional neural networks were successfully developed. The visual interpretation of the models' classification results was presented in the form of a heat map by using Grad-CAM. In the validation set, all models presented high classification accuracy (average accuracy = 0.852) and high classification precision (average precision = 0.846). Compared with the other three models, the NASNetL model had the highest classification accuracy (0.873) and the highest classification precision (0.867), and was the best model. The NASNetL model showed similar classification ability to the senior physician. Its classification accuracy was slightly lower than that of senior physician (0.881), and higher than that of junior physician (0.855). Meanwhile, it had good classification consistency with both senior physician (Kappa=0.712, P=0.000) and junior physician (Kappa=0.695, P=0.000).

Conclusion

·The endoscopic image classification models of Barrett's esophagus, developed by deep convolutional neural networks transfer learning, show good classification ability.

Key words： Barrett's esophagus; deep learning; transfer learning; gastrointestinal endoscopy; convolutional neural network

参考文献

1 郑荣寿, 孙可欣, 张思维, 等. 2015年中国恶性肿瘤流行情况分析[J]. 中华肿瘤杂志, 2019, 41(1): 19-28.
1 ZHENG R S, SUN K X, ZHANG S W, et al. Report of cancer epidemiology in China, 2015[J]. Chin J Oncol, 2019, 41(1): 19-28.
2 NUCCI D, MARINO A, REALDON S, et al. Lifestyle, WCRF/AICR recommendations, and esophageal adenocarcinoma risk: a systematic review of the literature[J]. Nutrients, 2021, 13(10): 3525.
3 ISHIHARA R, GODA K, OYAMA T. Endoscopic diagnosis and treatment of esophageal adenocarcinoma: introduction of Japan Esophageal Society classification of Barrett's esophagus[J]. J Gastroenterol, 2019, 54(1): 1-9.
4 HAMADE N, SHARMA P. Artificial intelligence in Barrett's esophagus[J]. Ther Adv Gastrointest Endosc, 2021, 14: 26317745211049964.
5 SHAHEEN N J, FALK G W, IYER P G, et al. ACG clinical guideline: diagnosis and management of Barrett's esophagus[J]. Am J Gastroenterol, 2016, 111(1): 30-50.
6 MANN R, GAJENDRAN M, PERISETTI A, et al. Advanced endoscopic imaging and interventions in GERD: an update and future directions[J]. Front Med (Lausanne), 2021, 8: 728696.
7 PAN W, LI X J, WANG W J, et al. Identification of Barrett's esophagus in endoscopic images using deep learning[J]. BMC Gastroenterol, 2021, 21(1): 479.
8 BORGLI H, THAMBAWITA V, SMEDSRUD P H, et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy[J]. Sci Data, 2020, 7(1): 283.
9 CHOLLET F. Xception: deep learning with depthwise separable convolutions[J/OL]. (2017-09-09)[2021-12-28]. http://arxiv.org/abs/1610.02357.
10 HE K, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[J/OL]. (2015-12-10)[2021-12-28]. http://arxiv.org/abs/1512.03385.
11 KOLESNIKOV A, BEYER L, ZHAI X H, et al. Big transfer (BiT): general visual representation learning[J/OL]. (2020-05-05)[2021-12-28]. http://arxiv.org/abs/1912.11370.
12 ZOPH B, VASUDEVAN V, SHLENS J, et al. Learning transferable architectures for scalable image recognition[J/OL]. (2017-12-07)[2021-12-30]. http://arxiv.org/abs/1707.07012v4.
13 HONG J S, PARK B Y, PARK H. Convolutional neural network classifier for distinguishing Barrett's esophagus and neoplasia endomicroscopy images[J]. Annu Int Conf IEEE Eng Med Biol Soc, 2017, 2017: 2892-2895.
14 DE GROOF A J, STRUYVENBERG M R, VAN DER PUTTEN J, et al. Deep-learning system detects neoplasia in patients with Barrett's esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking[J]. Gastroenterology, 2020, 158(4): 915-929.e4.
15 ALOM M Z, TAHA T M, YAKOPCIC C, et al. The history began from AlexNet: a comprehensive survey on deep learning approaches[J/OL]. (2018-09-12)[2021-12-27]. http://arxiv.org/abs/1803.01164.
16 SHIN H C, ROTH H R, GAO M C, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning[J]. IEEE Trans Med Imaging, 2016, 35(5): 1285-1298.
17 SAMALA R K, CHAN H P, HADJIISKI L, et al. Breast cancer diagnosis in digital breast tomosynthesis: effects of training sample size on multi-stage transfer learning using deep neural nets[J]. IEEE Trans Med Imaging, 2019, 38(3): 686-696.
Options
文章导航

/

〈 〉