CrossModal-CI

A cochlear implant (CI) is a neuro-prothesis that helps people with severe hearing loss to access sounds. This is achieved by transmitting sound vibrations into electrical stimulation that (re)activates cochlea cells and sends neural signals to the auditory cortex. However, electrical stimulation is artificial which does not provide natural and sufficient auditory inputs as in typically hearing individuals. Speech communication remains a huge challenge for CI recipients in day-to-day complex, noisy listening environments (at school/work, in streets/restaurants/train stations, meeting with families and friends etc). CI recipients often use visual speech cues (e.g., lip-reading) to compensate for these auditory barriers to help with communications. However, mechanisms underlying how their different sensory (auditory and visual) systems work together to integrate audiovisual speech are still unclear. It also remains unclear how interactions between these different systems change or 'reorganise' over time and how reorganisation relates to or predicts future speech comprehension outcomes.

Our Wellcome Trust funded ECA project (2025-2031, fEC: £1.6m) promises to conduct a series of cross-sectional and longitudinal experiments in adult CI recipients. We will combine brain imaging of EEG (real-time neural activity with millisecond precision) and high-density fNIRS/diffuse optical tomography (DOT) (measuring amount of blood oxygenation at specific brain regions) to measure CI recipients’ neural responses to audio and visual speech – how they differ from typical hearing, change over time and are modulated to improve speech comprehension. We will thus, scientifically, showcase example processes of cross-modal neuroplasticity (how sensory systems with different modalities, i.e., audition and vision, change the way they interact with each other in the brain) in humans with sensory impairment or deprivation after rehabilitation.

Wider implications, e.g., outcomes from multimodal imaging and clinical practicality (not just for CI recipients) may be expected. We may provide evidence for the potential power for monitoring and prognosis of speech and language comprehension outcomes using neuroimaging tools, e.g., EEG combined with portable and high-density fNIRS/DOT compared to tools commonly used but less physically or financially practical/accessible (e.g., MRI/fMRI) for treatments on a wide range of clinical individuals (children/infants who cannot stay still in the scanner, people implanted with electronic/metallic devices, those with physical disabilities or mental anxiety, in situations where large-scale (f)MRI screening is not considered as fiscally affordable or cost-effective, etc). This is particularly useful for many whose behavioural data are not easy to be reliably captured to reflect actual speech comprehension. We may also provide rigorous, consequential neuromodulation evidence for future techniques researchers could develop to support or supplement the existing speech and language therapy; in the longer run, for not only people with hard-of-hearing but also those with other conditions such as aphasia, developmental disorders, neurodivergence, or neurodegeneration.