Journal of Shanghai Jiao Tong University (Medical Science) >

2022 , Vol. 42 >Issue 5: 653 - 659

DOI: https://doi.org/10.3969/j.issn.1674-8115.2022.05.014

Techniques and methods

Development of endoscopic image classification models of Barrett's esophagus based on deep convolutional neural networks

  • Jiaxi LIN ,
  • Shengjia WANG ,
  • Xin ZHAO ,
  • Xin GAO ,
  • Minyue YIN ,
  • Jinzhou ZHU
Expand
  • 1.Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou 215006, China
    2.Suzhou Clinical Center of Digestive Disease, Jiangsu Province, Suzhou 215006, China
    3.Department of General Surgery, The First Affiliated Hospital of Soochow University, Suzhou 215006, China
ZHU Jinzhou, E-mail: jzzhu@zju.edu.cn.

Received date: 2022-01-07

  Accepted date: 2022-03-25

  Online published: 2022-05-07

Supported by

National Natural Science Foundation of China(82000540);Youth Program of Suzhou Health Committee(KJXW2019001)

Fold

Abstract

Objective

·To develop classification models for endoscopic images of Barrett's esophagus using deep convolutional neural network, and evaluate its classification performance.

Methods

·A total of 806 esophageal images were collected from Gastrointestinal Endoscopy Center of the First Affiliated Hospital of Soochow University and the HyperKvasir database, including 412 normal esophageal images and 394 Barrett's esophagus images. All the images were randomly divided into training set (85%) and validation set (15%). Four deep convolutional neural networks [Xception, NASNet Large (NASNetL), ResNet50V2 (ResNet) and BigTransfer (BiT)], which had been pre-trained in the ImageNet database, were used to perform transfer learning in the training set and develop endoscopic image classification models of Barrett's esophagus, respectively. The gradient-weighted class activation mapping (Grad-CAM) was used to visually interpret the classification results of the four models. In the validation set, the models' classification abilities were evaluated. Besides, to further evaluate the classification abilities of the models, the classification results of the validation set by senior and junior physicians were compared with the models'.

Results

·The 4 endoscopic image classification models of Barrett's esophagus based on deep convolutional neural networks were successfully developed. The visual interpretation of the models' classification results was presented in the form of a heat map by using Grad-CAM. In the validation set, all models presented high classification accuracy (average accuracy = 0.852) and high classification precision (average precision = 0.846). Compared with the other three models, the NASNetL model had the highest classification accuracy (0.873) and the highest classification precision (0.867), and was the best model. The NASNetL model showed similar classification ability to the senior physician. Its classification accuracy was slightly lower than that of senior physician (0.881), and higher than that of junior physician (0.855). Meanwhile, it had good classification consistency with both senior physician (Kappa=0.712, P=0.000) and junior physician (Kappa=0.695, P=0.000).

Conclusion

·The endoscopic image classification models of Barrett's esophagus, developed by deep convolutional neural networks transfer learning, show good classification ability.

Key words： Barrett's esophagus; deep learning; transfer learning; gastrointestinal endoscopy; convolutional neural network

Cite this article

Jiaxi LIN , Shengjia WANG , Xin ZHAO , Xin GAO , Minyue YIN , Jinzhou ZHU . Development of endoscopic image classification models of Barrett's esophagus based on deep convolutional neural networks[J]. Journal of Shanghai Jiao Tong University (Medical Science), 2022 , 42(5) : 653 -659 . DOI: 10.3969/j.issn.1674-8115.2022.05.014

References

1 郑荣寿, 孙可欣, 张思维, 等. 2015年中国恶性肿瘤流行情况分析[J]. 中华肿瘤杂志, 2019, 41(1): 19-28.
1 ZHENG R S, SUN K X, ZHANG S W, et al. Report of cancer epidemiology in China, 2015[J]. Chin J Oncol, 2019, 41(1): 19-28.
2 NUCCI D, MARINO A, REALDON S, et al. Lifestyle, WCRF/AICR recommendations, and esophageal adenocarcinoma risk: a systematic review of the literature[J]. Nutrients, 2021, 13(10): 3525.
3 ISHIHARA R, GODA K, OYAMA T. Endoscopic diagnosis and treatment of esophageal adenocarcinoma: introduction of Japan Esophageal Society classification of Barrett's esophagus[J]. J Gastroenterol, 2019, 54(1): 1-9.
4 HAMADE N, SHARMA P. Artificial intelligence in Barrett's esophagus[J]. Ther Adv Gastrointest Endosc, 2021, 14: 26317745211049964.
5 SHAHEEN N J, FALK G W, IYER P G, et al. ACG clinical guideline: diagnosis and management of Barrett's esophagus[J]. Am J Gastroenterol, 2016, 111(1): 30-50.
6 MANN R, GAJENDRAN M, PERISETTI A, et al. Advanced endoscopic imaging and interventions in GERD: an update and future directions[J]. Front Med (Lausanne), 2021, 8: 728696.
7 PAN W, LI X J, WANG W J, et al. Identification of Barrett's esophagus in endoscopic images using deep learning[J]. BMC Gastroenterol, 2021, 21(1): 479.
8 BORGLI H, THAMBAWITA V, SMEDSRUD P H, et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy[J]. Sci Data, 2020, 7(1): 283.
9 CHOLLET F. Xception: deep learning with depthwise separable convolutions[J/OL]. (2017-09-09)[2021-12-28]. http://arxiv.org/abs/1610.02357.
10 HE K, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[J/OL]. (2015-12-10)[2021-12-28]. http://arxiv.org/abs/1512.03385.
11 KOLESNIKOV A, BEYER L, ZHAI X H, et al. Big transfer (BiT): general visual representation learning[J/OL]. (2020-05-05)[2021-12-28]. http://arxiv.org/abs/1912.11370.
12 ZOPH B, VASUDEVAN V, SHLENS J, et al. Learning transferable architectures for scalable image recognition[J/OL]. (2017-12-07)[2021-12-30]. http://arxiv.org/abs/1707.07012v4.
13 HONG J S, PARK B Y, PARK H. Convolutional neural network classifier for distinguishing Barrett's esophagus and neoplasia endomicroscopy images[J]. Annu Int Conf IEEE Eng Med Biol Soc, 2017, 2017: 2892-2895.
14 DE GROOF A J, STRUYVENBERG M R, VAN DER PUTTEN J, et al. Deep-learning system detects neoplasia in patients with Barrett's esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking[J]. Gastroenterology, 2020, 158(4): 915-929.e4.
15 ALOM M Z, TAHA T M, YAKOPCIC C, et al. The history began from AlexNet: a comprehensive survey on deep learning approaches[J/OL]. (2018-09-12)[2021-12-27]. http://arxiv.org/abs/1803.01164.
16 SHIN H C, ROTH H R, GAO M C, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning[J]. IEEE Trans Med Imaging, 2016, 35(5): 1285-1298.
17 SAMALA R K, CHAN H P, HADJIISKI L, et al. Breast cancer diagnosis in digital breast tomosynthesis: effects of training sample size on multi-stage transfer learning using deep neural nets[J]. IEEE Trans Med Imaging, 2019, 38(3): 686-696.
Options
Outlines

/

〈 〉