论著 · 技术与方法

基于深度学习的结直肠息肉内镜图像分割和分类方法比较

  • 陈健 ,
  • 王珍妮 ,
  • 夏开建 ,
  • 王甘红 ,
  • 刘罗杰 ,
  • 徐晓丹
展开
  • 1.江苏省常熟市第一人民医院(苏州大学附属常熟医院)消化内科,常熟 215500
    2.江苏省常熟市医学人工智能与大数据重点实验室,常熟 215500
    3.江苏省常熟市中医院消化内科,常熟 215500
陈 健(1987—),男,副主任医师,硕士;电子信箱:szcsdocter@gmail.com
刘罗杰,电子信箱:Luojie13542@163.com
徐晓丹,电子信箱:xxddocter@gmail.com

收稿日期: 2024-01-01

  录用日期: 2024-03-15

  网络出版日期: 2024-06-28

基金资助

苏州市第二十三批科技发展计划项目(SLT2023006);常熟市医学人工智能与大数据重点实验室能力提升项目(CYZ202301);苏州市护理学会科研项目(SZHL-B-202407)

Comparative study on methods for colon polyp endoscopic image segmentation and classification based on deep learning

  • Jian CHEN ,
  • Zhenni WANG ,
  • Kaijian XIA ,
  • Ganhong WANG ,
  • Luojie LIU ,
  • Xiaodan XU
Expand
  • 1.Department of Gastroenterology, Changshu No. 1 People's Hospital (Changshu Hospital Affiliated to Soochow University), Jiangsu Province, Changshu 215500, China
    2.Changshu Key Laboratory of Medical Artificial Intelligence and Big Data, Jiangsu Province, Changshu 215500, China
    3.Department of Gastroenterology, Changshu Traditional Chinese Medicine Hospital, Jiangsu Province, Changshu 215500, China
LIU Luojie, E-mail:Luojie13542@163.com. #Corresponding authors.
XU Xiaodan, E-mail: xxddocter@gmail.com

Received date: 2024-01-01

  Accepted date: 2024-03-15

  Online published: 2024-06-28

Supported by

Suzhou City's 23rd Science and Technology Development Plan(SLT2023006);Changshu City Key Laboratory of Medical Artificial Intelligence and Big Data Capability Enhancement Project(CYZ202301);Scientific Research Project of Suzhou Nursing Association(SZHL-B-202407)

摘要

目的·比较不同深度学习方法在结直肠息肉内镜图像分割和分类任务中的性能,以确定最优方法。方法·从3家医院采集4个结肠息肉数据集,涵盖1 534个静态图像及15个肠镜视频。所有样本均经病理学验证,分为锯齿状病变和腺瘤性息肉2类。使用LabelMe工具进行多边形标注,将标注结果转换为整数掩膜格式。数据用于训练不同架构的深度神经网络,包括卷积神经网络、Transformer以及这2种技术的融合,建立有效的语义分割模型。对比不同架构模型自动诊断结肠息肉的多项性能指标,包括平均交并比(mIoU)、整体准确率(aAcc)、平均准确率(mAcc)、平均Dice系数(mDice)、平均F分数(mFscore)、平均精确率(mPrecision)和平均召回率(mRecall)。结果·开发了4种不同架构的语义分割模型,包括2种深度卷积神经网络架构(Fast-SCNN和DeepLabV3plus)、1种Transformer架构(Segformer)以及1种混合架构(KNet)。在对291张测试图像进行综合性能评估中,KNet最高mIoU为84.59%,显著优于Fast-SCNN(75.32%)、DeepLabV3plus(78.63%)和Segformer(80.17%)。在“背景”“锯齿状病变”和“腺瘤性息肉”3个类别上,KNet的交并比(IoU)分别为98.91%、74.12%和80.73%,均超越其他模型。KNet在关键性能指标上也表现优异,其中aAcc、mAcc、mDice、mFscore和mRecall分别达到98.59%、91.24%、91.31%、91.31%和91.24%,均优于其他模型。尽管在mPrecision上,91.46%并非最突出,但KNet的整体性能仍领先。在80张外部测试图像的推理测试中,KNet保持了81.53%的mIoU,展现出良好的泛化能力。结论·利用基于KNet混合架构的深度神经网络构建的结直肠息肉内镜图像语义分割模型表现出优异的预测性能,具有成为检测结直肠息肉高效工具的潜力。

本文引用格式

陈健 , 王珍妮 , 夏开建 , 王甘红 , 刘罗杰 , 徐晓丹 . 基于深度学习的结直肠息肉内镜图像分割和分类方法比较[J]. 上海交通大学学报(医学版), 2024 , 44(6) : 762 -772 . DOI: 10.3969/j.issn.1674-8115.2024.06.012

Abstract

Objective ·To compare the performance of various deep learning methods in the segmentation and classification of colorectal polyp endoscopic images, and identify the most effective approach. Methods ·Four colorectal polyp datasets were collected from three hospitals, encompassing 1 534 static images and 15 videos. All samples were pathologically validated and categorized into two types: serrated lesions and adenomatous polyps. Polygonal annotations were performed by using the LabelMe tool, and the annotated results were converted into integer mask formats. These data were utilized to train various architectures of deep neural networks, including convolutional neural network (CNN), Transformers, and their fusion, aiming to develop an effective semantic segmentation model. Multiple performance indicators for automatic diagnosis of colon polyps by different architecture models were compared, including mIoU, aAcc, mAcc, mDice, mFscore, mPrecision and mRecall. Results ·Four different architectures of semantic segmentation models were developed, including two deep CNN architectures (Fast-SCNN and DeepLabV3plus), one Transformer architecture (Segformer), and one hybrid architecture (KNet). In a comprehensive performance evaluation of 291 test images, KNet achieved the highest mIoU of 84.59%, significantly surpassing Fast-SCNN (75.32%), DeepLabV3plus (78.63%), and Segformer (80.17%). Across the categories of “background”, “serrated lesions” and “adenomatous polyps” , KNet's intersection over union (IoU) were 98.91%, 74.12%, and 80.73%, respectively, all exceeding other models. Additionally, KNet performed excellently in key performance metrics, with aAcc, mAcc, mDice, mFscore, and mRecall reaching 98.59%, 91.24%, 91.31%, 91.31%, and 91.24%, respectively, all superior to other models. Although its mPrecision of 91.46% was not the most outstanding, KNet's overall performance remained leading. In inference testing on 80 external test images, KNet maintained an mIoU of 81.53%, demonstrating strong generalization capabilities. Conclusion ·The semantic segmentation model of colorectal polyp endoscopic images constructed by deep neural network based on KNet hybrid architecture, exhibits superior predictive performance, demonstrating its potential as an efficient tool for detecting colorectal polyps.

参考文献

1 GUNTER M J, ALHOMOUD S, ARNOLD M, et al. Meeting report from the joint IARC-NCI international cancer seminar series: a focus on colorectal cancer[J]. Ann Oncol, 2019, 30(4): 510-519.
2 中华人民共和国国家卫生健康委员会, 中华医学会肿瘤学分会. 中国结直肠癌诊疗规范(2023年版)[J]. 中华外科杂志, 2023, 61(8):617-644.
2 National Health Commission of the People's Republic of China, Chinese Society of Oncology. Chinese protocol of diagnosis and treatment of colorectal cancer (2023 edition)[J]. Chinese Journal of Surgery, 2023, 61(8): 617-644.
3 GUPTA S, LIEBERMAN D, ANDERSON J C, et al. Recom-mendations for follow-up after colonoscopy and polypectomy: a consensus update by the US multi-society task force on colorectal cancer[J]. Gastrointest Endosc, 2020, 91(3): 463-485.e5.
4 LIEBERMAN D A, REX D K, WINAWER S J, et al. Guidelines for colonoscopy surveillance after screening and polypectomy: a consensus update by the US Multi-Society Task Force on Colorectal Cancer[J]. Gastroenterology, 2012, 143(3): 844-857.
5 BRETTHAUER M, KALAGER M, ADAMI H O. Do's and don'ts in evaluation of endoscopic screening for gastrointestinal cancers[J]. Endoscopy, 2016, 48(1): 75-80.
6 MORI Y, KUDO S E, BERZIN T M, et al. Computer-aided diagnosis for colonoscopy[J]. Endoscopy, 2017, 49(8): 813-819.
7 LEUFKENS A M, VAN OIJEN M G, VLEGGAAR F P, et al. Factors influencing the miss rate of polyps in a back-to-back colonoscopy study[J]. Endoscopy, 2012, 44(5): 470-475.
8 KIM N H, JUNG Y S, JEONG W S, et al. Miss rate of colorectal neoplastic polyps and risk factors for missed polyps in consecutive colonoscopies[J]. Intest Res, 2017, 15(3): 411-418.
9 RUTTER M D, BEINTARIS I, VALORI R, et al. World endoscopy organization consensus statements on post-colonoscopy and post-imaging colorectal cancer[J]. Gastroenterology, 2018, 155(3): 909-925.e3.
10 GONG E J, BANG C S, LEE J J, et al. No-code platform-based deep-learning models for prediction of colorectal polyp histology from white-light endoscopy images: development and performance verification[J]. J Pers Med, 2022, 12(6): 963.
11 GARCíA-RODRíGUEZ A, TUDELA Y, CóRDOVA H, et al. In vivo computer-aided diagnosis of colorectal polyps using white light endoscopy[J]. Endosc Int Open, 2022, 10(9): E1201-E1207.
12 WANG P, LIU P, GLISSEN BROWN J R, et al. Lower adenoma miss rate of computer-aided detection-assisted colonoscopy vs routine white-light colonoscopy in a prospective tandem study[J]. Gastroenterology, 2020, 159(4): 1252-1261.e5.
13 HASSAN C, EAST J, RADAELLI F, et al. Bowel preparation for colonoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline-Update 2019[J]. Endoscopy, 2019, 51(8): 775-794.
14 RUSSELL B C, TORRALBA A, MURPHY K P, et al. LabelMe: a database and web-based tool for image annotation[J]. Int J Comput Vis, 2008, 77(1): 157-173.
15 URBAN G, TRIPATHI P, ALKAYALI T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy[J]. Gastroenterology, 2018, 155(4): 1069-1078.e8.
16 CHEUNG T H, YEUNG D Y. A survey of automated data augmentation for image classification: learning to compose, mix, and generate[J]. IEEE Trans Neural Netw Learn Syst, 2023. DOI: 10. 1109/TNNLS.2023.3282258.
17 SHIN H C, ROTH H R, GAO M, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning[J]. IEEE Trans Med Imaging, 2016, 35(5): 1285-1298.
18 LESLIE A, CAREY F A, PRATT N R, et al. The colorectal adenoma-carcinoma sequence[J]. Br J Surg, 2002, 89(7): 845-860.
19 ZHOU Y J, LU X F, CHEN H, et al. Single-cell transcriptomics reveals early molecular and immune alterations underlying the serrated neoplasia pathway toward colorectal cancer[J]. Cell Mol Gastroenterol Hepatol, 2023, 15(2): 393-424.
20 GOTO H, ODA Y, MURAKAMI Y, et al. Proportion of de novo cancers among colorectal cancers in Japan[J]. Gastroenterology, 2006, 131(1): 40-46.
21 KIM K M, LEE E J, HA S, et al. Molecular features of colorectal hyperplastic polyps and sessile serrated adenoma/polyps from Korea[J]. Am J Surg Pathol, 2011, 35(9): 1274-1286.
22 LUI R N, SUNG J J Y. Sessile serrated adenoma/polyps: why we should be working flat out to understand more about these flat lesions?[J]. J Gastroenterol Hepatol, 2019, 34(10): 1667-1668.
23 NAGTEGAAL I D, ODZE R D, KLIMSTRA D, et al. The 2019 WHO classification of tumours of the digestive system[J]. Histopathology, 2020, 76(2): 182-188.
24 JHA D, SMEDSRUD P, RIEGLER M, et al. ResUNet++: an advanced architecture for medical image segmentation[R]. San Diego: IEEE International Symposium on Multimedia, 2019.
25 CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans Pattern Anal Mach Intell, 2018, 40(4): 834-848.
26 HSU C M, HSU C C, HSU Z M, et al. Colorectal polyp image detection and classification through grayscale images and deep learning[J]. Sensors (Basel), 2021, 21(18): 5995.
27 LO C M, YEH Y H, TANG J H, et al. Rapid polyp classification in colonoscopy using textural and convolutional features[J]. Healthcare (Basel), 2022, 10(8): 1494.
28 KRENZER A, HEIL S, FITTING D, et al. Automated classification of polyps using deep learning architectures and few-shot learning[J]. BMC Med Imaging, 2023, 23(1): 59.
29 LIU Z, LV Q, YANG Z, et al. Recent progress in transformer-based medical image analysis[J]. Comput Biol Med, 2023, 164: 107268.
30 ZHANG W, PANG J, CHEN K, et al. K-Net: towards unified image segmentation [R]. Montreal: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021.
文章导航

/