基于Transformer和扩散模型的头颅侧位片颈椎分割方法在正畸临床中的初步应用

doi:10.3969/j.issn.1674-8115.2024.12.011

上海交通大学学报（医学版） ›› 2024, Vol. 44 ›› Issue (12): 1579-1586.doi: 10.3969/j.issn.1674-8115.2024.12.011

• 论著 · 技术与方法 • 上一篇

基于Transformer和扩散模型的头颅侧位片颈椎分割方法在正畸临床中的初步应用

刘洋¹(), 吴梦怡²(), 胡尧², 亓坤³, 王渝彬², 赵悦²(), 宋锦璘¹()

^1.重庆医科大学附属口腔医院正畸科，重庆 401147
^2.重庆邮电大学通信与信息工程学院，重庆 400065
^3.西安交通大学口腔医院正畸科，西安 710004

收稿日期:2024-03-05 接受日期:2024-08-21 出版日期:2024-12-28 发布日期:2024-12-28
通讯作者: 赵悦,宋锦璘 E-mail:yangliu@hospital.cqmu.edu.cn;zhaoyue@cqupt.edu.cn;songjinlin@hospital.cqmu.edu.cn
作者简介:刘洋（1987—），男，副研究员，博士；电子信箱：yangliu@hospital.cqmu.edu.cn
刘洋（1987—），男，副研究员，博士；电子信箱：yangliu@hospital.cqmu.edu.cn第一联系人：（刘洋、吴梦怡并列第一作者）
基金资助:
国家自然科学基金(62206036);重庆市自然科学基金(CSTB2023NSCQ-MSX1065);中华口腔医学会正畸专委会青年人才培养项目(COS-B2021-07);重庆市科卫联合项目(2023QNXM021);中国牙病防治基金会项目(A2023-012);重庆市教育委员会科学技术研究项目(KJQN202300407)

Preliminary application of a cervical vertebra segmentation method based on Transformer and diffusion model for lateral cephalometric radiographs in orthodontic clinical practice

LIU Yang¹(), WU Mengyi²(), HU Yao², QI Kun³, WANG Yubin², ZHAO Yue²(), SONG Jinlin¹()

^1.Department of Orthodontics, Stomatological Hospital of Chongqing Medical University, Chongqing 401147, China
^2.School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
^3.Department of Orthodontics, Stomatological Hospital of Xi'an Jiaotong University, Xi'an 710004, China

Received:2024-03-05 Accepted:2024-08-21 Online:2024-12-28 Published:2024-12-28
Contact: ZHAO Yue,SONG Jinlin E-mail:yangliu@hospital.cqmu.edu.cn;zhaoyue@cqupt.edu.cn;songjinlin@hospital.cqmu.edu.cn
Supported by:
National Natural Science Foundation of China(62206036);Natural Science Foundation of Chongqing(CSTB2023NSCQ-MSX1065);Youth Talent Training Program of the Orthodontics Special Committee of Chinese Stomatological Association(COS-B2021-07);Chongqing Science and Health Joint Medical Research Project(2023QNXM021);China Oral Health Foundation(A2023-012);Scientific and Technological Research Program of Chongqing Municipal Education Commission(KJQN202300407)

摘要/Abstract

摘要：

目的·针对错畸形生长发育高峰期骨骼形态变化复杂、难以精准评估的临床难点，利用扩散模型与Transformer深度学习算法构建颈椎图像分割模型并评估其分割性能。方法·使用基于Transformer与扩散模型相结合的方法对185例正畸患者（44例来自重庆医科大学附属口腔医院，141例来自西安交通大学口腔医院）的头颅侧位片进行精准的颈椎分割。首先对图像进行预处理，裁剪出感兴趣的颈椎骨区域，随机将所有数据划分为训练集（79.6%）和测试集（20.4%）。利用U-Net构成的扩散模型和条件模型进行特征提取，引入Transformer模块学习噪声和语义特征之间的相互作用。将多尺度图像进行融合，以增强低对比度图像中的细微结构和边界纹理特征。将该方法与U-Net和SOLOv2方法进行比较，通过Dice相似系数（Dice similarity coefficient，DSC）、交并比（intersection over union，IoU）2项指标定量比较颈椎图像分割性能。通过医师的人工标注结果和模型可视化结果对分割性能进行定性评估。结果·基于Transformer的扩散模型颈椎图像分割方法的DSC和IoU评分分别达到93.3%和87.5%，明显优于U-Net和SOLOv2方法（在DSC上分别领先3.0%和4.1%，在IoU上分别领先5.2%和7.1%）。尽管单张图像的处理时间较长，但分割精度显著提升。相较于U-Net和SOLOv2，基于Transformer的扩散模型颈椎图像分割方法在处理复杂、低对比度和边界模糊的图像时表现出更高的稳定性和鲁棒性，能够精准分割出颈椎骨的清晰边界和完整结构。结论·基于Transformer的扩散模型颈椎图像分割网络能够增强颈椎图像中的边缘和纹理特征，更容易识别不同椎骨的边界，从而获得自动、准确、稳健的颈椎分割结果，可辅助颈椎骨成熟度分析。

关键词: 扩散模型, 颈椎分割, 深度学习, 头颅侧位片

Abstract:

Objective ·To construct a cervical vertebra image segmentation model by using a diffusion model with the Transformer deep learning algorithm, and evaluate its segmentation performance, to address the clinical challenge of accurately assessing complex changes in skeletal morphology during the growth and developmental peaks of malocclusion. Methods ·Accurate cervical vertebra segmentation was performed on cephalometric radiographs from 185 orthodontic patients (44 cases from the Stomatological Hospital of Chongqing Medical University and 141 cases from the Stomatological Hospital of Xi'an Jiaotong University) by using a method combining Transformer and diffusion models. First, the images were preprocessed to crop out the cervical vertebra region of interest, and all data were randomly divided into a training set (79.6%) and a test set (20.4%). The diffusion model and a conditional model based on U-Net were utilized for feature extraction, with a Transformer module introduced to learn the interaction between noise and semantic features. Multi-scale images were fused to enhance fine structure and boundary texture features in low-contrast images. The proposed method was compared with U-Net and SOLOv2 methods. The segmentation performance was quantitatively evaluated by two metrics, Dice Similarity Coefficient (DSC) and Intersection over Union (IoU), and also qualitatively assessed through physicians' manual annotations and model visualization results. Results ·The cervical vertebra segmentation method based on Transformer and diffusion models achieved DSC and IoU scores of 93.3% and 87.5%, respectively, significantly outperforming the U-Net and SOLOv2 methods (with improvements of 3.0% and 4.1% in DSC, and 5.2% and 7.1% in loU, respectively). Despite the longer processing time for a single image, segmentation accuracy was significantly improved. Compared with U-Net and SOLOv2, the proposed method also showed higher stability and robustness in processing complex, low-contrast and blurred-boundary images, and was able to accurately segment the cervical vertebrae with clear boundaries and complete structures. Conclusion ·The Transformer-based diffusion model for cervical vertebra segmentation can enhance the edge and texture features in cervical vertebra images and recognize the boundaries of different vertebrae more easily. Thus, automatic, accurate, and robust cervical vertebra segmentation results are achieved, which can assist in cervical vertebral maturation analysis.

Key words: diffusion model, cervical vertebra segmentation, deep learning, lateral cephalometric radiograph

中图分类号:

R783.5

刘洋, 吴梦怡, 胡尧, 亓坤, 王渝彬, 赵悦, 宋锦璘. 基于Transformer和扩散模型的头颅侧位片颈椎分割方法在正畸临床中的初步应用[J]. 上海交通大学学报（医学版）, 2024, 44(12): 1579-1586.

LIU Yang, WU Mengyi, HU Yao, QI Kun, WANG Yubin, ZHAO Yue, SONG Jinlin. Preliminary application of a cervical vertebra segmentation method based on Transformer and diffusion model for lateral cephalometric radiographs in orthodontic clinical practice[J]. Journal of Shanghai Jiao Tong University (Medical Science), 2024, 44(12): 1579-1586.

图/表 4

图1 头颅侧位片图像和手动注释Note： Three samples, designated from A to C, were randomly selected from 191 cases of manually annotated lateral cephalometric radiographs by radiologists. In these samples, C2, C3, and C4 corresponded to the second, third, and fourth vertebrae of the cervical spines, respectively.

Fig 1 Lateral cephalometric radiographs and manual annotations

图2 基于Transformer的扩散模型颈椎分割网络Note： A. Overall architecture of the MedSegDiff-V2 network. The network took raw cranial lateral slice data as input, and proceeded to obtain results pertaining to the segmentation of the cervical vertebrae. It was achieved through a five-step process. ①Image preprocessing. The region of interest (ROI) was cropped and resized to 256×256 pixels. ②Dataset splitting. The 191 images were divided into two sets: 152 for training and 39 for testing. ③Model training. The MedSegDiff-V2 model was trained using the training dataset. ④Image sampling. Images from the test dataset were sampled using the trained diffusion. ⑤Performance evaluation. The DSC and IoU formulas were used to calculate the corresponding metrics from the ground truth masks and model predictions of the test dataset. B. MedSegDiff-V2 architecture. A Transformer-based diffusion network for image segmentation. FFT—Fast Fourier Transform; IFFT—reverse operation of the FFT; MLP—multi-layer perceptron; U-SA—uncertain spatial attention; NBP-Filter—neural band-pass filter. C. Neural band-pass filter.

Fig 2 A Transformer-based cervical vertebra segmentation network for diffusion model

表1 不同颈椎分割方法的DSC、IoU评分和处理时间

Tab 1 DSC， mIoU and processing time of different cervical vertebra segmentation methods

Method	DSC/%	IoU/%	Processing time/s
U-Net	90.3	82.3	2.5
SOLOv2	89.2	80.4	1.7
Transformer-based diffusion model	93.3	87.5	32.1

图3 不同方法在5个示例上的分割结果和相应的人工标注Note： Five samples, designated from A to E, were randomly selected from the test set data of cervical vertebra segmentation images. The cervical vertebra segmentation method based on the Transformer and diffusion model was compared with the original input image, two additional methods (U-Net and SOLOv2), and the ground truth. The regions delineated by the dashed circles represent the areas where the segmentation results of the various methods differ from the ground truth.

Fig 3 Visualization of segmentation results obtained by different methods and the respective manual ground-truth annotations for five examples

参考文献 26

1	LI H R, LI H Z, YUAN L J, et al. The psc-CVM assessment system: a three-stage type system for CVM assessment based on deep learning[J]. BMC Oral Health, 2023, 23(1): 557.
2	FISHMAN L S. Chronological versus skeletal age, an evaluation of craniofacial growth[J]. Angle Orthod, 1979, 49(3): 181-189.
3	ALKHAL H A, WONG R W K, RABIE A B. Correlation between chronological age, cervical vertebral maturation and Fishman's skeletal maturity indicators in southern Chinese[J]. Angle Orthod, 2008, 78(4): 591-596.
4	GANDINI P, MANCINI M, ANDREANI F. A comparison of hand-wrist bone and cervical vertebral analyses in measuring skeletal maturation[J]. Angle Orthod, 2006, 76(6): 984-989.
5	BACCETTI T, FRANCHI L, JR MCNAMARA J A. An improved version of the cervical vertebral maturation (CVM) method for the assessment of mandibular growth[J]. Angle Orthod, 2002, 72(4): 316-323.
6	ASLAN M S, ALI A, RARA H, et al. An automated vertebra identification and segmentation in CT images[C]//2010 IEEE International Conference on Image Processing. Hong Kong, China: IEEE, 2010: 233-236.
7	LIM P H, BAGCI U, BAI L. Introducing Willmore flow into level set segmentation of spinal vertebrae[J]. IEEE Trans Biomed Eng, 2013, 60(1): 115-122.
8	YAO J H, BURNS J E, FORSBERG D, et al. A multi-center milestone study of clinical vertebral CT segmentation[J]. Comput Med Imaging Graph, 2016, 49: 16-28.
9	SHIM J H, KIM W S, KIM K G, et al. Evaluation of U-Net models in automated cervical spine and cranial bone segmentation using X-ray images for traumatic atlanto-occipital dislocation diagnosis[J]. Sci Rep, 2022, 12(1): 21438.
10	ZHANG F, ZHENG L Y, CHEN Y R, et al. Fully automatic cervical vertebrae segmentation via enhanced U²-Net[C]//2023 IEEE International Conference on Image Processing.Kuala Lumpur, Malaysia: IEEE, 2023: 2900-2904.
11	ZHANG L, WANG H. A novel segmentation method for cervical vertebrae based on PointNet++ and converge segmentation[J]. Comput Methods Programs Biomed, 2021, 200: 105798.
12	潘恩元, 钟原, 李平. 联邦异质性数据下半监督颈椎MRI分割模型[J]. 计算机工程, 2024, 50(9): 367-376.
	PAN E Y, ZHONG Y, LI P. Semi-supervised cervical spine MRI segmentation model in federated heterogeneous data[J]. Computer Engineering, 2024, 50(9): 367-376.
13	朱逸峰, 赵凯, 郭丽, 等. 基于深度学习模型实现颈椎MR图像上各结构的自动分割[J]. 放射学实践, 2021, 36(12): 1558-1562.
	ZHU Y F, ZHAO K, GUO L, et al. Automatic segmentation of cervical spine structures on MRI images based on deep learning: a preliminary study[J]. Radiology Practice, 2021, 36(12): 1558-1562.
14	李擎, 皇甫玉彬, 李江昀, 等. UConvTrans: 全局和局部信息交互的双分支心脏图像分割[J]. 上海交通大学学报, 2023, 57(5): 570-581.
	LI Q, HUANGFU Y B, LI J Y, et al. UConvTrans: a dual-flow cardiac image segmentation network by global and local information integration[J]. Journal of Shanghai Jiao Tong University, 2023, 57(5): 570-581.
15	张峻宁, 苏群星, 王成, 等. 一种改进变换网络的域自适应语义分割网络[J]. 上海交通大学学报, 2021, 55(9): 1158-1168.
	ZHANG J N, SU Q X, WANG C, et al. A domain adaptive semantic segmentation network based on improved transformation network[J]. Journal of Shanghai Jiao Tong University, 2021, 55(9): 1158-1168.
16	吕超凡, 言颖杰, 林力, 等. 基于点云语义分割算法的下颌角截骨面设计[J]. 上海交通大学学报, 2022, 56(11): 1509-1517.
	LÜ C F, YAN Y J, LIN L, et al. Design of mandibular angle osteotomy plane based on point cloud semantic segmentation algorithm[J]. Journal of Shanghai Jiao Tong University, 2022, 56(11): 1509-1517.
17	DONG Z W, YUAN G J, HUA Z, et al. Diffusion model-based text-guided enhancement network for medical image segmentation[J]. Expert Syst Appl, 2024, 249: 123549.
18	LI G J, JIN D H, ZHENG Y J, et al. A generic plug & play diffusion-based denosing module for medical image segmentation[J]. Neural Netw, 2024, 172: 106096.
19	ZHAO Y Y, LI J J, REN L, et al. DTAN: diffusion-based Text Attention Network for medical image segmentation[J]. Comput Biol Med, 2024, 168: 107728.
20	GUO X T, YANG Y W, YE C F, et al. Accelerating diffusion models via pre-segmentation diffusion sampling for medical image segmentation[C]//2023 IEEE 20th International Symposium on Biomedical Imaging. Cartagena, Colombia: IEEE, 2023: 1-5.
21	WU J D, JI W, FU H Z, et al. MedSegDiff-V2: diffusion based medical image segmentation with Transformer[EB/OL]. (2023-12-24) [2024-06-19]. http://arxiv.org/abs/2301.11798.
22	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. Cham, Switzerland: Springer, 2015: 234-241.
23	WANG X, ZHANG R, KONG T, et al. SOLOv2: dynamic and fast instance segmentation[C]//Advances in Neural Information Processing System 33 (NeurIPS 2020).Vancouver, Canada: NeurIPS, 2020: 17721-17732.
24	HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[EB/OL].(2023-12-24) [2024-06-19]. http://arxiv.org/abs/2006.11239.
25	SONG J M, MENG C L, ERMON S. Denoising diffusion implicit models[EB/OL].(2022-10-05) [2024-06-19].http://arxiv.org/abs/2010.02502.
26	DUAN Z J, WANG C Y, CHEN C, et al. Optimal linear subspace search: learning to construct fast and high-quality schedulers for diffusion models[EB/OL].(2023-08-11) [2024-06-19].http://arxiv.org/abs/2305.14677.

[1]	陈健, 王珍妮, 夏开建, 王甘红, 刘罗杰, 徐晓丹. 基于深度学习的结直肠息肉内镜图像分割和分类方法比较[J]. 上海交通大学学报（医学版）, 2024, 44(6): 762-772.
[2]	赵珂珂, 蒋蓓蓓, 张璐, 王凌云, 张亚平, 解学乾. 超低剂量平扫CT深度学习图像重建评价肺部病灶的可行性[J]. 上海交通大学学报（医学版）, 2022, 42(8): 1062-1069.
[3]	林嘉希, 汪盛嘉, 赵鑫, 高欣, 殷民月, 朱锦舟. 基于深度卷积神经网络的Barrett食管内镜图片分类模型的建立[J]. 上海交通大学学报（医学版）, 2022, 42(5): 653-659.
[4]	陈立奇, 薛卓维, 吴氢凯. 基于磁共振成像的女性盆底器官三维数字模型重建的研究进展[J]. 上海交通大学学报（医学版）, 2022, 42(3): 381-386.
[5]	李小敏, 曲扬, 张少霆, 赵亮, 刘畅, 谢帅宁, 戴尅戎, 艾松涛. 人工智能技术在骨肌系统影像学方面的应用[J]. 上海交通大学学报(医学版), 2021, 41(2): 262-266.
[6]	陆言巧，沈兰，何奔. 人工智能在心血管疾病的辅助诊疗中的应用[J]. 上海交通大学学报（医学版）, 2020, 40(2): 259-.

基于Transformer和扩散模型的头颅侧位片颈椎分割方法在正畸临床中的初步应用

Preliminary application of a cervical vertebra segmentation method based on Transformer and diffusion model for lateral cephalometric radiographs in orthodontic clinical practice

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 26

相关文章 6

编辑推荐

Metrics