收稿日期: 2024-03-05
录用日期: 2024-08-21
网络出版日期: 2024-12-28
基金资助
国家自然科学基金(62206036);重庆市自然科学基金(CSTB2023NSCQ-MSX1065);中华口腔医学会正畸专委会青年人才培养项目(COS-B2021-07);重庆市科卫联合项目(2023QNXM021);中国牙病防治基金会项目(A2023-012);重庆市教育委员会科学技术研究项目(KJQN202300407)
Preliminary application of a cervical vertebra segmentation method based on Transformer and diffusion model for lateral cephalometric radiographs in orthodontic clinical practice
Received date: 2024-03-05
Accepted date: 2024-08-21
Online published: 2024-12-28
Supported by
National Natural Science Foundation of China(62206036);Natural Science Foundation of Chongqing(CSTB2023NSCQ-MSX1065);Youth Talent Training Program of the Orthodontics Special Committee of Chinese Stomatological Association(COS-B2021-07);Chongqing Science and Health Joint Medical Research Project(2023QNXM021);China Oral Health Foundation(A2023-012);Scientific and Technological Research Program of Chongqing Municipal Education Commission(KJQN202300407)
目的·针对错畸形生长发育高峰期骨骼形态变化复杂、难以精准评估的临床难点,利用扩散模型与Transformer深度学习算法构建颈椎图像分割模型并评估其分割性能。方法·使用基于Transformer与扩散模型相结合的方法对185例正畸患者(44例来自重庆医科大学附属口腔医院,141例来自西安交通大学口腔医院)的头颅侧位片进行精准的颈椎分割。首先对图像进行预处理,裁剪出感兴趣的颈椎骨区域,随机将所有数据划分为训练集(79.6%)和测试集(20.4%)。利用U-Net构成的扩散模型和条件模型进行特征提取,引入Transformer模块学习噪声和语义特征之间的相互作用。将多尺度图像进行融合,以增强低对比度图像中的细微结构和边界纹理特征。将该方法与U-Net和SOLOv2方法进行比较,通过Dice相似系数(Dice similarity coefficient,DSC)、交并比(intersection over union,IoU)2项指标定量比较颈椎图像分割性能。通过医师的人工标注结果和模型可视化结果对分割性能进行定性评估。结果·基于Transformer的扩散模型颈椎图像分割方法的DSC和IoU评分分别达到93.3%和87.5%,明显优于U-Net和SOLOv2方法(在DSC上分别领先3.0%和4.1%,在IoU上分别领先5.2%和7.1%)。尽管单张图像的处理时间较长,但分割精度显著提升。相较于U-Net和SOLOv2,基于Transformer的扩散模型颈椎图像分割方法在处理复杂、低对比度和边界模糊的图像时表现出更高的稳定性和鲁棒性,能够精准分割出颈椎骨的清晰边界和完整结构。结论·基于Transformer的扩散模型颈椎图像分割网络能够增强颈椎图像中的边缘和纹理特征,更容易识别不同椎骨的边界,从而获得自动、准确、稳健的颈椎分割结果,可辅助颈椎骨成熟度分析。
刘洋 , 吴梦怡 , 胡尧 , 亓坤 , 王渝彬 , 赵悦 , 宋锦璘 . 基于Transformer和扩散模型的头颅侧位片颈椎分割方法在正畸临床中的初步应用[J]. 上海交通大学学报(医学版), 2024 , 44(12) : 1579 -1586 . DOI: 10.3969/j.issn.1674-8115.2024.12.011
Objective ·To construct a cervical vertebra image segmentation model by using a diffusion model with the Transformer deep learning algorithm, and evaluate its segmentation performance, to address the clinical challenge of accurately assessing complex changes in skeletal morphology during the growth and developmental peaks of malocclusion. Methods ·Accurate cervical vertebra segmentation was performed on cephalometric radiographs from 185 orthodontic patients (44 cases from the Stomatological Hospital of Chongqing Medical University and 141 cases from the Stomatological Hospital of Xi'an Jiaotong University) by using a method combining Transformer and diffusion models. First, the images were preprocessed to crop out the cervical vertebra region of interest, and all data were randomly divided into a training set (79.6%) and a test set (20.4%). The diffusion model and a conditional model based on U-Net were utilized for feature extraction, with a Transformer module introduced to learn the interaction between noise and semantic features. Multi-scale images were fused to enhance fine structure and boundary texture features in low-contrast images. The proposed method was compared with U-Net and SOLOv2 methods. The segmentation performance was quantitatively evaluated by two metrics, Dice Similarity Coefficient (DSC) and Intersection over Union (IoU), and also qualitatively assessed through physicians' manual annotations and model visualization results. Results ·The cervical vertebra segmentation method based on Transformer and diffusion models achieved DSC and IoU scores of 93.3% and 87.5%, respectively, significantly outperforming the U-Net and SOLOv2 methods (with improvements of 3.0% and 4.1% in DSC, and 5.2% and 7.1% in loU, respectively). Despite the longer processing time for a single image, segmentation accuracy was significantly improved. Compared with U-Net and SOLOv2, the proposed method also showed higher stability and robustness in processing complex, low-contrast and blurred-boundary images, and was able to accurately segment the cervical vertebrae with clear boundaries and complete structures. Conclusion ·The Transformer-based diffusion model for cervical vertebra segmentation can enhance the edge and texture features in cervical vertebra images and recognize the boundaries of different vertebrae more easily. Thus, automatic, accurate, and robust cervical vertebra segmentation results are achieved, which can assist in cervical vertebral maturation analysis.
1 | LI H R, LI H Z, YUAN L J, et al. The psc-CVM assessment system: a three-stage type system for CVM assessment based on deep learning[J]. BMC Oral Health, 2023, 23(1): 557. |
2 | FISHMAN L S. Chronological versus skeletal age, an evaluation of craniofacial growth[J]. Angle Orthod, 1979, 49(3): 181-189. |
3 | ALKHAL H A, WONG R W K, RABIE A B. Correlation between chronological age, cervical vertebral maturation and Fishman's skeletal maturity indicators in southern Chinese[J]. Angle Orthod, 2008, 78(4): 591-596. |
4 | GANDINI P, MANCINI M, ANDREANI F. A comparison of hand-wrist bone and cervical vertebral analyses in measuring skeletal maturation[J]. Angle Orthod, 2006, 76(6): 984-989. |
5 | BACCETTI T, FRANCHI L, JR MCNAMARA J A. An improved version of the cervical vertebral maturation (CVM) method for the assessment of mandibular growth[J]. Angle Orthod, 2002, 72(4): 316-323. |
6 | ASLAN M S, ALI A, RARA H, et al. An automated vertebra identification and segmentation in CT images[C]//2010 IEEE International Conference on Image Processing. Hong Kong, China: IEEE, 2010: 233-236. |
7 | LIM P H, BAGCI U, BAI L. Introducing Willmore flow into level set segmentation of spinal vertebrae[J]. IEEE Trans Biomed Eng, 2013, 60(1): 115-122. |
8 | YAO J H, BURNS J E, FORSBERG D, et al. A multi-center milestone study of clinical vertebral CT segmentation[J]. Comput Med Imaging Graph, 2016, 49: 16-28. |
9 | SHIM J H, KIM W S, KIM K G, et al. Evaluation of U-Net models in automated cervical spine and cranial bone segmentation using X-ray images for traumatic atlanto-occipital dislocation diagnosis[J]. Sci Rep, 2022, 12(1): 21438. |
10 | ZHANG F, ZHENG L Y, CHEN Y R, et al. Fully automatic cervical vertebrae segmentation via enhanced U2-Net[C]//2023 IEEE International Conference on Image Processing.Kuala Lumpur, Malaysia: IEEE, 2023: 2900-2904. |
11 | ZHANG L, WANG H. A novel segmentation method for cervical vertebrae based on PointNet++ and converge segmentation[J]. Comput Methods Programs Biomed, 2021, 200: 105798. |
12 | 潘恩元, 钟原, 李平. 联邦异质性数据下半监督颈椎MRI分割模型[J]. 计算机工程, 2024, 50(9): 367-376. |
12 | PAN E Y, ZHONG Y, LI P. Semi-supervised cervical spine MRI segmentation model in federated heterogeneous data[J]. Computer Engineering, 2024, 50(9): 367-376. |
13 | 朱逸峰, 赵凯, 郭丽, 等. 基于深度学习模型实现颈椎MR图像上各结构的自动分割[J]. 放射学实践, 2021, 36(12): 1558-1562. |
13 | ZHU Y F, ZHAO K, GUO L, et al. Automatic segmentation of cervical spine structures on MRI images based on deep learning: a preliminary study[J]. Radiology Practice, 2021, 36(12): 1558-1562. |
14 | 李擎, 皇甫玉彬, 李江昀, 等. UConvTrans: 全局和局部信息交互的双分支心脏图像分割[J]. 上海交通大学学报, 2023, 57(5): 570-581. |
14 | LI Q, HUANGFU Y B, LI J Y, et al. UConvTrans: a dual-flow cardiac image segmentation network by global and local information integration[J]. Journal of Shanghai Jiao Tong University, 2023, 57(5): 570-581. |
15 | 张峻宁, 苏群星, 王成, 等. 一种改进变换网络的域自适应语义分割网络[J]. 上海交通大学学报, 2021, 55(9): 1158-1168. |
15 | ZHANG J N, SU Q X, WANG C, et al. A domain adaptive semantic segmentation network based on improved transformation network[J]. Journal of Shanghai Jiao Tong University, 2021, 55(9): 1158-1168. |
16 | 吕超凡, 言颖杰, 林力, 等. 基于点云语义分割算法的下颌角截骨面设计[J]. 上海交通大学学报, 2022, 56(11): 1509-1517. |
16 | Lü C F, YAN Y J, LIN L, et al. Design of mandibular angle osteotomy plane based on point cloud semantic segmentation algorithm[J]. Journal of Shanghai Jiao Tong University, 2022, 56(11): 1509-1517. |
17 | DONG Z W, YUAN G J, HUA Z, et al. Diffusion model-based text-guided enhancement network for medical image segmentation[J]. Expert Syst Appl, 2024, 249: 123549. |
18 | LI G J, JIN D H, ZHENG Y J, et al. A generic plug & play diffusion-based denosing module for medical image segmentation[J]. Neural Netw, 2024, 172: 106096. |
19 | ZHAO Y Y, LI J J, REN L, et al. DTAN: diffusion-based Text Attention Network for medical image segmentation[J]. Comput Biol Med, 2024, 168: 107728. |
20 | GUO X T, YANG Y W, YE C F, et al. Accelerating diffusion models via pre-segmentation diffusion sampling for medical image segmentation[C]//2023 IEEE 20th International Symposium on Biomedical Imaging. Cartagena, Colombia: IEEE, 2023: 1-5. |
21 | WU J D, JI W, FU H Z, et al. MedSegDiff-V2: diffusion based medical image segmentation with Transformer[EB/OL]. (2023-12-24) [2024-06-19]. http://arxiv.org/abs/2301.11798. |
22 | RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. Cham, Switzerland: Springer, 2015: 234-241. |
23 | WANG X, ZHANG R, KONG T, et al. SOLOv2: dynamic and fast instance segmentation[C]//Advances in Neural Information Processing System 33 (NeurIPS 2020).Vancouver, Canada: NeurIPS, 2020: 17721-17732. |
24 | HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[EB/OL].(2023-12-24) [2024-06-19]. http://arxiv.org/abs/2006.11239. |
25 | SONG J M, MENG C L, ERMON S. Denoising diffusion implicit models[EB/OL].(2022-10-05) [2024-06-19].http://arxiv.org/abs/2010.02502. |
26 | DUAN Z J, WANG C Y, CHEN C, et al. Optimal linear subspace search: learning to construct fast and high-quality schedulers for diffusion models[EB/OL].(2023-08-11) [2024-06-19].http://arxiv.org/abs/2305.14677. |
/
〈 |
|
〉 |