听觉脑干植入声码器模型的开发及验证

doi:10.3969/j.issn.1674-8115.2024.10.010

上海交通大学学报（医学版） ›› 2024, Vol. 44 ›› Issue (10): 1279-1286.doi: 10.3969/j.issn.1674-8115.2024.10.010

• 论著 · 技术与方法 • 上一篇

听觉脑干植入声码器模型的开发及验证

张钦杰¹^,²(), 黄穗³, 谭皓月¹^,², 周祥¹^,², 王君怡¹^,², 刘雨滋¹^,², 文雯¹^,², 郭嘉¹^,², 吴皓¹^,²(), 贾欢¹^,²()

^1.上海交通大学医学院附属第九人民医院耳鼻咽喉头颈外科，上海 200011
^2.上海交通大学医学院耳科学研究所，上海市耳鼻疾病转化医学重点实验室，上海 200125
^3.浙江诺尔康神经电子科技股份有限公司，杭州 311100

收稿日期:2024-02-29 接受日期:2024-04-25 出版日期:2024-10-28 发布日期:2024-10-28
通讯作者: 吴皓,贾欢 E-mail:zqj0727@sjtu.edu.cn;wuhao@shsmu.edu.cn;huan.jia.orl@shsmu.edu.cn
作者简介:张钦杰（1998—），男，硕士生；电子信箱: zqj0727@sjtu.edu.cn。
基金资助:
上海市耳鼻疾病转化医学重点实验室项目(14DZ2260300);上海市黄浦区产业扶持基金(XK2019015);上海市人才发展基金(2019047);上海交通大学医学院转化医学协同创新项目(TM202011)

Establishment and verification of auditory brainstem implant vocoder model

ZHANG Qinjie¹^,²(), HUANG Sui³, TAN Haoyue¹^,², ZHOU Xiang¹^,², WANG Junyi¹^,², LIU Yuzi¹^,², WEN Wen¹^,², GUO Jia¹^,², WU Hao¹^,²(), JIA Huan¹^,²()

^1.Department of Otolaryngology-Head and Neck Surgery, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200011, China
^2.Ear Institute, Shanghai Jiao Tong University School of Medicine; Shanghai Key Laboratory of Translational Medicine on Ear and Nose Diseases, Shanghai 200125, China
^3.Nurotron Biotechnology Co. , Ltd. , Hangzhou 311100, China

Received:2024-02-29 Accepted:2024-04-25 Online:2024-10-28 Published:2024-10-28
Contact: WU Hao,JIA Huan E-mail:zqj0727@sjtu.edu.cn;wuhao@shsmu.edu.cn;huan.jia.orl@shsmu.edu.cn
Supported by:
Program of Shanghai Key Laboratory of Translational Medicine on Ear and Nose Diseases(14DZ2260300);Shanghai Huangpu District Industrial Support Fund(XK2019015);Shanghai Talent Development Fund(2019047);Collaborative Innovation Project for Translational Medicine at Shanghai Jiao Tong University School of Medicine(TM202011)

摘要/Abstract

摘要：

目的·基于人工耳蜗（cochlear implant，CI）声码器及听觉脑干植入（auditory brainstem implant，ABI）电极阵列排布特征，开发ABI声码器并验证其可靠性。方法·通过MATLAB软件构建“n-of-m”编码策略的CI/ABI声码器。每一帧内仅选取能量最大的n个通道的包络，其中串扰系数（interaction coefficient，IC）（范围1~3）、通道数（范围5~22）、电极阵列拓扑模式（CI/ABI）为可调节参数，可合成仿真语音。招募正常听力受试者进行闭合式仿真音素的测听，并将音素识别准确率（元音20题/条件，辅音11题/条件）与参考文献中对应条件的CI及ABI植入者的音素准确率进行比较，明确声码器的IC值并验证其可靠性。结果·声码器可成功合成所有测试用仿真声。IC2、IC3 2个条件的仿真元音及辅音识别准确率与参考文献相应数据比较，差异均无统计学意义（P>0.05）；IC2与参考文献中元/辅音准确率的差值较IC3与参考文献数据的差值更小（元音|d|：1.6% vs. 20% ；辅音|d|：8.4% vs. 9.9%），确定本模型的最优IC为2。修改电极阵列拓扑模式为ABI，显示16通道ABI仿真音素识别准确率显著低于16通道CI。5~8通道间的ABI仿真元/辅音正确率比较，差异无统计学意义（P>0.05），与参考文献结论一致。结论·成功建立了基于“n-of-m”编码策略的CI/ABI声码器，并确定最佳IC。建立的ABI声码器经心理声学实验评估可靠性较高，可为ABI专属编码策略的模型验证提供合适的技术手段。

关键词: 听觉脑干植入, 声码器, 音素识别, 心理声学, 电极阵列拓扑

Abstract:

Objective ·To develope an auditory brainstem implant (ABI) vocoder based on cochlear implant (CI) vocoder characteristics and ABI electrode array topology, and to verify its reliability. Methods ·An "n-of-m" coding strategy CI/ABI vocoder was constructed based on MATLAB. Within each frame, only the envelopes of the n channels with the highest energy were selected. The interaction coefficient (IC) (range: 1?3), channel numbers (range: 5?22), and electrode array topology (CI/ABI) were adjustable parameters, allowing for the synthesis of simulated speech. Psychoacoustic evaluation was employed, recruiting normal hearing subjects to perform closed-set simulated phoneme perception. The phoneme recognition accuracy (20 vowel questions/condition, 11 consonant questions/condition) was compared with the corresponding conditions of CI and ABI from reference literature to determine the IC value of the vocoder and verify its reliability. Results ·The vocoder successfully synthesized all test stimuli. In the closed-set CI-simulated speech recognition, the simulated vowel and consonant recognition accuracy for IC2 and IC3 conditions showed no significant difference compared to the accuracy reported in the CI reference literature (P>0.05). The difference in vowel and consonant accuracy between IC2 and the literature was smaller than that between IC3 and the literature (vowel |d|=1.6% vs. 20%, consonant |d|=8.4% vs. 9.9%), thus determining the optimal interaction coefficient of this model as 2. Subsequently, when modifying the electrode array topology to ABI, it was found that the simulated phoneme recognition accuracy for a 16-channel ABI was significantly lower than that for the 16-channel CI group, consistent with the reported literature. The simulated vowel and consonant accuracy within the 5?8 channel range for ABI showed no significant difference (P>0.05), also aligning with the trend reported in the literature. Conclusion ·A CI/ABI vocoder based on "n-of-m" coding strategy is established and the optimal IC is determined. The established ABI encoder has been evaluated for high reliability through psychoacoustic experiments. It provides suitable technical means for validating ABI-specific coding strategies.

Key words: auditory brainstem implant, vocoder, phoneme recognition, psychoacoustic, electrode array topology

中图分类号:

R764.4

张钦杰, 黄穗, 谭皓月, 周祥, 王君怡, 刘雨滋, 文雯, 郭嘉, 吴皓, 贾欢. 听觉脑干植入声码器模型的开发及验证[J]. 上海交通大学学报（医学版）, 2024, 44(10): 1279-1286.

ZHANG Qinjie, HUANG Sui, TAN Haoyue, ZHOU Xiang, WANG Junyi, LIU Yuzi, WEN Wen, GUO Jia, WU Hao, JIA Huan. Establishment and verification of auditory brainstem implant vocoder model[J]. Journal of Shanghai Jiao Tong University (Medical Science), 2024, 44(10): 1279-1286.

图/表 4

图1 “n-of-m”语音信号处理方案原理示意图Note： AM— amplitude modulation.

Fig 1 Block diagram of the "n-of-m" strategy

图2 “n-of-m”语音信号处理方案仿真情况

Fig 2 Block diagram of computer simulation of the "n-of-m" strategy

图3 通道串扰示意图Note： A. Nurotron CI (Nurotron-Standard). B. ABI (WH-01A). Red lines indicate channel interaction.

Fig 3 Channel Interaction diagram

图4 正常听力人群CI仿真音素识别结果Note： ①P=0.022. Ref—reference.

Fig 4 Phoneme recognition of normal listeners to CI-simulated audios

参考文献 32

1	WROBEL C, ZAFEIRIOU M P, MOSER T. Understanding and treating paediatric hearing impairment[J]. EBioMedicine, 2021, 63: 103171.
2	GARCIA A, HALEEM A, POE S, et al. Auditory brainstem implant outcomes in tumor andNontumor patients: a systematic review[J]. Otolaryngol Head Neck Surg, 2024, 170(6): 1648-1658.
3	WONG K, KOZIN E D, KANUMURI V V, et al. Auditory brainstem implants: recent progress and future perspectives[J]. Front Neurosci, 2019, 13: 10.
4	VAN DER STRAATEN T F K, NETTEN A P, BOERMANS P P B M, et al. Pediatric auditory brainstem implant users compared with cochlear implant users with additional disabilities[J]. and, 2019, 40(7): 936-945.
5	GÄRTNER L, LENARZ T, BÜCHNER A. Measurements of the local evoked potential from the cochlear nucleus in patients with an auditory brainstem implant and its implication to auditory perception and audio processor programming[J]. PLoS One, 2021, 16(4): e0249535.
6	MCINTURFF S, COEN F V, HIGHT A E, et al. Comparison of responses to DCN vs. VCN stimulation in a mouse model of the auditory brainstem implant (ABI)[J]. J Assoc Res Otolaryngol, 2022, 23(3): 391-412.
7	VACHICOURAS N, TARABICHI O, KANUMURI V V, et al. Microstructured thin-film electrode technology enables proof of concept of scalable, soft auditory brainstem implants[J]. Sci Transl Med, 2019, 11(514): eaax9487.
8	AZADPOUR M, SHAPIRO W H, ROLAND J T Jr, et al. Assessing temporal responsiveness of primary stimulated neurons in auditory brainstem and cochlear implant users[J]. Hear Res, 2021, 401: 108163.
9	MCKAY C M, AZADPOUR M, JAYEWARDENE-ASTON D, et al. Electrode selection and speech understanding in patients with auditory brainstem implants[J]. Ear Hear, 2015, 36(4): 454-463.
10	GOEHRING T, ARCHER-BOYD A W, ARENBERG J G, et al. The effect of increased channel interaction on speech perception with cochlear implants[J]. Sci Rep, 2021, 11(1): 10383.
11	CREW J D, GALVIN J J 3rd, FU Q J. Channel interaction limits melodic pitch perception in simulated cochlear implants[J]. J Acoust Soc Am, 2012, 132(5): EL429-EL435.
12	LAMPING W, DEEKS J M, MAROZEAU J, et al. The effect of phantom stimulation and pseudomonophasic pulse shapes on pitch perception by cochlear implant listeners[J]. J Assoc Res Otolaryngol, 2020, 21(6): 511-526.
13	PARTOUCHE E, ADENIS V, STAHL P, et al. What is the benefit of ramped pulse shapes for activating auditory cortex neurons? an electrophysiological study in an animal model of cochlear implant[J]. Brain Sci, 2023, 13(2): 250.
14	SHANNON R V, ZENG F G, KAMATH V, et al. Speech recognition with primarily temporal cues[J]. Science, 1995, 270(5234): 303-304.
15	KONG F H, ZHOU H L, MO Y F, et al. Comparable encoding, comparable perceptual pattern: acoustic and electric hearing[J]. IEEE Trans Neural Syst Rehabil Eng, 2023, 31: 2326-2337.
16	杨一威, 徐月晋, 缪吉昌, 等. 人工耳蜗的膜电位积分放电刺激方案及其数字信号处理[J]. 南方医科大学学报, 2012, 32(10): 1435-1439.
	YANG Y W, XU Y J, MIU J C, et al. Digital signal processing of a novel neuron discharge model stimulation strategy for cochlear implants [J]. Journal of Southern Medical University, 2012, 32(10): 1435-1439.
17	孟强, 田岚, 徐东平, 等. 全相位带通滤波器用于听觉重建的参数优化研究[J]. 复旦学报(自然科学版), 2020, 59(5): 551-557.
	MENG Q, TIAN L, XU D P, et al. Research on parameter optimization of all-phase band-pass filter for auditory reconstruction [J]. Journal of Fudan University(Natural Science), 2020, 59(5): 551-557.
18	KUCHTA J, OTTO S R, SHANNON R V, et al. The multichannel auditory brainstem implant: how many electrodes make sense?[J]. J Neurosurg, 2004, 100(1): 16-23.
19	NELSON D A, DONALDSON G S, KREFT H. Forward-masked spatial tuning curves in cochlear implant users[J]. J Acoust Soc Am, 2008, 123(3):1522-1543.
20	CHEN X Q, YOU Y Y, YANG J, et al. Effects of nonlinear frequency compression on Mandarin speech and sound-quality perception in hearing-aid users[J]. Int J Audiol, 2020, 59(7): 524-533.
21	YANG J, QIAN J Y, CHEN X Q, et al. Effects of nonlinear frequency compression on the acoustic properties and recognition of speech sounds in Mandarin Chinese[J]. J Acoust Soc Am, 2018, 143(3): 1578.
22	QI S, CHEN X Q, YANG J, et al. Effects of adaptive non-linear frequency compression in hearing aids on mandarin speech and sound-quality perception[J]. Front Neurosci, 2021, 15: 722970.
23	FAYAD J N, OTTO S R, SHANNON R V, et al. Cochlear and brainstem auditory prostheses "neural interface for hearing restoration: cochlear and brain stem implants"[J]. Proc IEEE, 2008, 96(7): 1085-1095.
24	吴皓, 贾欢. 关注人工听觉脑干植入[J]. 中华医学杂志, 2021, 101(2): 92-96.
	WU H, JIA H. Auditory brainstem implantation: current status and prospects[J]. National Medical Journal of China, 2021, 101(2): 92-96.
25	吴皓. 人工听觉植入最新进展[J]. 中华耳鼻咽喉头颈外科杂志, 2023, 58(Suppl): 13-20.
	WU H. The latest progress in artificial auditory implantation[J]. Chinese Journal of Otorhinolaryngology Head and Neck Surgery, 2023, 58(Suppl): 13-20.
26	LI X, NIE K B, IMENNOV N S, et al. Improved perception of music with a harmonic based algorithm for cochlear implants[J]. IEEE Trans Neural Syst Rehabil Eng, 2013, 21(4): 684-694.
27	贾欢, 陈颖, 张治华, 等. 人工听觉脑干植入在先天性耳聋低龄儿童中的应用探索[J]. 上海交通大学学报(医学版), 2020, 40(10): 1324-1329.
	JIA H, CHEN Y, ZHANG Z H, et al. Auditory brainstem implantation in young children with congenital deafness: a case report[J]. Journal of Shanghai Jiaotong University (Medical Science), 2020, 40(10): 1324-1329.
28	SADDLER M R, GONZALEZ R, MCDERMOTT J H. Deep neural network models reveal interplay of peripheral coding and stimulus statistics in pitch perception[J]. Nat Commun, 2021, 12(1): 7278.
29	BINGABR M, ESPINOZA-VARAS B, LOIZOU P C. Simulating the effect of spread of excitation in cochlear implants[J]. Hear Res, 2008, 241(1/2): 73-79.
30	SCHMID G, UBERBACHER R, SAMARAS T, et al. High-resolution numerical model of the middle and inner ear for a detailed analysis of radio frequency absorption[J]. Phys Med Biol, 2007, 52(7): 1771-1781.
31	D'ALESSANDRO S, HANDLER M, SABA R, et al. Computer simulation of the electrical stimulation of the human vestibular system: effects of the reactive component of impedance on voltage waveform and nerve selectivity[J]. J Assoc Res Otolaryngol, 2022, 23(6): 815-833.
32	周祥, 潘金锡, 张钦杰, 等. 听觉脑干植入豚鼠模型构建的标准化步骤及评价[J]. 上海交通大学学报(医学版), 2022, 42(5): 583-590.
	ZHOU X, PAN J X, ZHANG Q J, et al. Establishment and evaluation of standardized steps for building a guinea pig model of auditory brainstem implantation[J]. Journal of Shanghai Jiaotong University(Medical Science), 2022, 42(5): 583-590.

[1]	周祥, 潘金锡, 张钦杰, 李蕴, 陈颖, 谭皓月, 彭飞, 黄穗, 谭治平, 吴皓, 贾欢. 听觉脑干植入豚鼠模型构建的标准化步骤及评价[J]. 上海交通大学学报（医学版）, 2022, 42(5): 583-590.
[2]	贾欢1, 2，陈颖1, 2，张治华1, 2，李静洁3，李蕴1, 2，陈见清1, 2，李孛1, 2，谭皓月1, 2，汪照炎1, 2，吴皓1, 2. 人工听觉脑干植入在先天性耳聋低龄儿童中的应用探索[J]. 上海交通大学学报(医学版), 2020, 40(10): 1324-1329.

听觉脑干植入声码器模型的开发及验证

Establishment and verification of auditory brainstem implant vocoder model

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 32

相关文章 2

编辑推荐

Metrics