Research Papers

Front. Hum. Neurosci., 21 September 2022
Sec. Brain-Computer Interfaces
Volume 16 – 2022 | https://doi.org/10.3389/fnhum.2022.864983

This article is part of the Research Topic

Mind over brain, brain over mind: cognitive causes and consequences of controlling brain activity – Volume II

View all 4 Articles 

Recruitment and Differential Firing Patterns of Single Units During Conditioning to a Tone in a Mute Locked-In Human

Philip Kennedy1* and Andre J. Cervantes2

• 1Neural Signals, Inc., Duluth, GA, United States
• 2Belize International Institute of Neuroscience, Belize City, Belize

Single units that are not related to the desired task can become related to the task by conditioning their firing rates. We theorized that, during conditioning of firing rates to a tone, (a) unrelated single units would be recruited to the task; (b) the recruitment would depend on the phase of the task; (c) tones of different frequencies would produce different patterns of single unit recruitment. In our mute locked-in participant, we conditioned single units using tones of different frequencies emitted from a tone generator. The conditioning task had three phases: Listen to the tone for 20 s, then silently sing the tone for 10 s, with a prior control period of resting for 10 s. Twenty single units were recorded simultaneously while feedback of one of the twenty single units was made audible to the mute locked-in participant. The results indicate that (a) some of the non-audible single units were recruited during conditioning, (b) some were recruited differentially depending on the phase of the paradigm (listen, rest, or silent sing), and (c) single unit firing patterns were specific for different tone frequencies such that the tone could be recognized from the pattern of single unit firings. These data are important when conditioning single unit firings in brain-computer interfacing tasks because they provide evidence that increased numbers of previously unrelated single units can be incorporated into the task. This incorporation expands the bandwidth of the recorded single unit population and thus enhances the brain-computer interface. This is the first report of conditioning of single unit firings in a human participant with a brain to computer implant.

1. Ganesh1. A.J. Cervantes2., and P.R. Kennedy1..
2. A. Ganesh, Neural Signals Inc, Neural Prostheses Laboratory, 3400 McClure Bridge Road, Suite D402, Duluth, GA 30096, USA.
3. A.J.Cervantes, MD, Belize International Institute of Neuroscience, 4760 University Heights, West Landivar, Belize City, Belize.
4. P.R. Kennedy, PhD, (Corresponding author), Neural Signals Inc., Neural Prostheses Laboratory, 3400 McClure Bridge Road, Suite D402, Duluth, GA 30096, USA. 

Kennedy is the corresponding author. Email: Phlkennedy@aol.com

This paper is original research. 

Front Hum Neurosci. 2022 Aug 3;16:874199. doi: 10.3389/fnhum.2022.874199. eCollection 2022. PMID: 35992944 Free PMC article. 

Running title: Slow firing single units are essential for optimal decoding of silent speech

Summary

The motivation of someone who is locked-in, that is, paralyzed and mute, is to find relief for their loss of function. The data presented in this report is part of an attempt to restore one of those lost functions, namely, speech. An essential feature of the development of a speech prosthesis is optimal decoding of patterns of recorded neural signals during silent or covert speech, that is, speaking ‘inside the head’ with output that is inaudible due to the paralysis of the articulators. The aim of this paper is to illustrate the importance of both fast and slow single unit firings recorded from an individual with locked-in syndrome and from an intact participant speaking silently. Long duration electrodes were implanted in the motor speech cortex for up to 13 years in the locked-in participant. The data herein provide evidence that slow firing single units are essential for optimal decoding accuracy. Additional evidence indicates that slow firing single units can be conditioned in the locked-in participant five years after implantation, further supporting their role in decoding.

Philip Kennedy MD PhD

Submitted to Brain-Computer Interface Special Edition 2019.

ABSTRACT

The development of an invasive speech prosthetic requires five steps: (1) Chronically recorded and stable single units over the life time of the locked-in subject; (2) cosmetically acceptable and fully implantable neural signal amplifiers and transmitters; (3) a rapid data processing system that operates at a near conversational rate with latency of a few hundred milliseconds from neural activity to speech output; (4) speech outputted from the processor; (5) and all these components mounted on a wheelchair. The present paper deals with item (3): Deep learning paradigms from Matlab are used to process the single units into words and phonemes. The present results demonstrate classification of the words and phonemes, and, more importantly, demonstrate fitting of the words and phonemes to Target (or stored) words and phonemes. This Fitting (or matching) paradigm is the basis of the rapid processing system that is essential to produce near conversational speech.

Marla Gearing PhD¹, Philip Kennedy MD PhD²

Submitted to Frontiers in Neuroscience 2019.

ABSTRACT

Aims: Electrodes that provide brain to machine or computer interfacing must survive the lifetime of the person to be an acceptable prosthetic. The electrodes may be external such as with Electroencephalographic (EEG), internal extracortical such as Electrocorticographic (ECoG) or intracortical.

Methods: Most intracortical electrodes are placed close to the neuropil being recorded and do not survive years of recording. However, the Neurotrophic Electrode (NE) is placed within the cortex and the neuropil grows inside and through the hollow tip of the electrode and is thus trapped inside. Highly flexible coiled lead wires minimize the strain on the electrode tip. Postmortem histological analysis includes immunohistochemical detection of neurofilaments and the absence of gliosis.

Results: This NE configuration led to a decade long recording in this locked-in person. At year nine, the neural activity underwent conditioning experiments indicating that the neural activity was not noise. This paper presents data on the histological analysis of the tissue inside the electrode tip after 13 years of implantation.

Conclusion: The histological analysis laid out herein is strong evidence that the brain can grow neurites into the electrode tip and record for a decade. This is profoundly important in the field of brain to machine or computer interfacing by implying that long term electrodes should incorporate some means of growing the neuropil into the electrode rather than simply placing the electrode into the neuropil.

¹ Departments of Pathology and Laboratory Medicine and Neurology, Emory University School of Medicine, Atlanta, GA 30322
² Neural Signals Inc., Duluth, GA 30096

P.R. Kennedy PhD¹, A.J. Cervantes MD², P. Ehirim³, and C. Gambrell¹. 

Book chapter in Brain-Machine Interfaces: Uses and Developments 2018.

ABSTRACT

Advances in the development of a speech prosthesis are reported in this chapter. Two patients have been implanted: a locked-in, mute and paralyzed male in 2004 (ER) and a speaking male in 2014 (PK). ER’s data has been reported in various papers [10-14] including functional neural data at year nine (in press, 16). Building on ER’s data, motor and sensory relationships are described for subject PK, as well as the results of detecting phonemes, words and phrases during overt and covert speech. Different techniques are described including using single unit data and single unit bursts, both from averaged and individual trials. A neural net Fitting program from Matlab using bursting units more reliably detects words and phrases than traditional single unit rate analysis. Once the Fitting program is trained there is minimal delay in detection of words of phrases which is required for near-conversational speech. A roadmap is described that outlines the steps needed to provide speech to mute individuals.

¹ Neural Signals Inc., Duluth, GA, US
² Neurosurgical Clinic, Belize City, Belize
³ Neurosurgical Brain and Spine, Gwinnett Medical Center, Lawrenceville, GA, US

Philip R. Kennedy¹, Dinal S. Andreasen², Jess Bartels¹, Princewill Ehirim³, E. Joe Wright¹, Steven Seibert¹, Andre Joel Cervantes⁴

Handbook of Biomedical Engineering. 2018

ABSTRACT

Background: The development of a reliable neural interface is essential for lifetime cortical control of prosthetic devices such as robotic arms, paralyzed limbs or speech. Standard tine or wire electrodes are not long lasting, surviving a few years with very few remaining useful signals.

New Method: The Neurotrophic Electrode engages radically different methodology that allows the brain’s neuropil to grow into the electrode tip. Six human subjects have been implanted with this electrode, with the longest lasting implant being functional for over 10 years. Eight monkey and 42 rat implants provided valuable early experience that gained FDA approval.

Results: Successful anchoring of the electrode tip within the neuropil has resulted in functionally usable single unit recordings for over decade. The data presented here include task related cross correlations of neuronal ensembles and reciprocal conditioning of firing rates.

Comparison with Existing Methods: Tine and wire type electrodes lose units over months and years unlike the Neurotrophic
Electrode described here.

Conclusions: These data demonstrate that stable recordings can be accomplished in humans by allowing neuropil to grow into the electrode, rather than by inserting the electrode into the neuropil. This is the first electrode methodology to produce such long lasting signals that remain functional for over a decade.

Keywords: Long-term recording; Neurotrophic Electrode; Multi-units; single units; speech prosthesis

¹ Neural Signals Inc., Duluth, GA, USA.
² Georgia Tech Research Institute, Atlanta, GA USA.
³ Gwinnett Medical Center, Lawrenceville, GA, USA.
⁴ Neurosurgical and Spinal Services Associates, Belize City, Belize

F.H. Guenther^1,2, J.S. Brumberg^1,3, E.J. Wright³, A. Nieto-Castanon⁴, J.A. Tourville¹, M. Panko¹, R. Law¹, S.A. Siebert³, J.L. Bartels³, D.S. Andreasen^3,5, P. Ehirim⁶, H. Mao⁷, and P.R. Kennedy³

PLoS ONE. (2009) 9;4(12):e8218

ABSTRACT

Background: Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfortunately they are only capable of slow communication rates. In the current study we use a novel approach to speech restoration in which we decode continuous auditory parameters for a real-time speech synthesizer from neuronal activity in motor cortex during attempted speech.

Methodology/Principal Findings: Neural signals recorded by a Neurotrophic Electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome, characterized by near-total paralysis with spared cognition, were transmitted wirelessly across the scalp and used to drive a speech synthesizer. A Kalman filter-based decoder translated the neural signals generated during attempted speech into continuous parameters for controlling a synthesizer that provided immediate (within 50 ms) auditory feedback of the decoded sound. Accuracy of the volunteer’s vowel productions with the synthesizer improved quickly with practice, with a 25% improvement in average hit rate (from 45% to 70%) and 46% decrease in average endpoint error from the first to the last block of a three-vowel task.

Conclusions/Significance: Our results support the feasibility of neural prostheses that may have the potential to provide near-conversational synthetic speech output for individuals with severely impaired speech motor control. They also provide an initial glimpse into the functional properties of neurons in speech motor cortical areas.

¹ Department of Cognitive and Neural Systems, Boston University, 677 Beacon Street, Boston, MA 02115
² Division of Health Sciences and Technology, Harvard University-Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139
³ Neural Signals Inc., 3400 McClure Bridge Road, Duluth, GA 30096
⁴ StatsANC LLC, Ayacucho 48 7A, Capital Federal 1025, Buenos Aires, Argentina
⁵ Georgia Tech Research Institute, 5247 Forest Brook Parkway, Marietta, GA 30068
⁶ Gwinnett Medical Center, 500 Professional Blvd, Suite 200, Lawrenceville, GA 30345
⁷ Emory Center for Systems Imaging, Emory Univ. Hosp., 1364 Clifton Road, NE, Atlanta, GA 30322

Jonathan S. Brumberg¹, E. Joe Wright², Dinal S. Andreasen^2,3, Frank H. Guenther^4,1,5 and Philip R. Kennedy²

Frontiers in Neuroscience 5(65)1-14

ABSTRACT

Half the phonemes in the English language have been detected in our subject:

We conducted a neurophysiological study of attempted speech production in a paralyzed human volunteer using chronic microelectrode recordings. The volunteer suffers from locked-in syndrome leaving him in a state of near-total paralysis, though he maintains good cognition and sensation. In this study, we investigated the feasibility of supervised classification techniques for prediction of intended phoneme production in the absence of any overt movements including speech. Such classification or decoding ability has the potential to greatly improve the quality-of-life of many people who are otherwise unable to speak by providing a direct communicative link to the general community. We examined the performance of three classifiers on a multi-class discrimination problem in which the items were 38 American English phonemes including mono-phthong and diphthong vowels and consonants. The three classifiers differed in performance, but averaged between 16-21% overall accuracy (chance-level is 1/38 or 2.6%). Further, the distribution of phonemes classified statistically above chance was non-uniform though 20 of 38 phonemes were classified with statistical significance for all three classifiers. These preliminary results suggest supervised classification techniques are capable of performing large scale multi-class discrimination for attempted speech production and may provide the basis for future communication prostheses.

Brett Matthews, Mark Clements

Asilomar conference, Asilomar, CA, November 2011.

ABSTRACT

Our collaborators continue to decode the data

This paper discusses recent work in automatic spike sorting for continuous extra-cellular cortical traces. Our spike sorting framework jointly models neuronal firing times and corresponding action potential waveforms as a discrete-state latent variable process. We model the likelihood of the observed firing occurrence times as the aggregation of multiple hidden point processes based on inter-arrival probability distributions. We evaluate our method on two real, continuous, partially labeled recordings of extracellular traces from rat hippocampus obtaining total error rates (false positives + false negatives) of 5.60% and 1.86% in clean conditions, outperforming both a Gaussian mixture model (GMM) baseline and the state-of-the art WaveClus method. Our method continues to outperform in the presence of added noise on the same data. We then perform an empirical study of two free parameters for our method on a semi-artificial dataset. We find that our method is more sensitive to parameter tuning in more difficult data and noise conditions.

Philip Kennedy¹,Dinal Andreasen^1,2, Jess Bartels¹, Princewill Ehirim⁴, Hui Mao⁵,Meel Velliste^1,3, Thomas Wichmann⁶, Joe Wright¹

Proceedings in Brain Research, Ch.1, August 2011.

ABSTRACT

Further results with the electrode in our mute subject:

A reliable neural interface that lasts a lifetime will lead to the development of neural prosthetic devices as well as the possibility that brain function can be enhanced. Our data demonstrate that a reliable neural interface is best achieved when the surrounding neuropil grows into the electrode tip where it is held securely, allowing myelinated axons to be recorded using implanted amplifiers. Stable single and multi-units were recorded from three implanted subjects and classified according to amplitudes and firing rates. In one paralyzed and mute subject implanted for over five years with a double electrode in the speech motor cortex, the single units allowed recognition of over half the 39 English language phonemes detected using a variety of decoding methods. These single units were used by the subject in a speech task where vowel phonemes were recognized and fed back to the subject using audio output. Weeks of training resulted in an 80% success rate in producing four vowels in an adaptation of the classic center-out task used in motor control studies. The importance of using single units was shown in a different task using pure tones that the same subject heard and then sung or hummed in his head. Feedback was associated with smoothly coordinated unit firings. The plasticity of the unit firings was demonstrated over several sessions first without, and then with, feedback. These data suggest that units can be reliably recorded over years, that there is an inverse relationship between single unit firing rate and amplitude, that pattern recognition decoding paradigms can allow phoneme recognition, that single units appear more important than multi-units when precision is important and that units are plastic in their functional relationships. These characteristics of a reliable neural interface are essential for the development of neural prostheses and also for the future enhancement of human brain function.

¹ Neural Signals Inc., 3400 McClure Bridge Road, Duluth, GA
² Georgia Institute of Technology, Atlanta, GA
³ Dept. of Neurobiology, University of Pittsburgh, Pittsburgh, PA
⁴ Dept. Neurosurgery, Gwinnett Medical Center, Lawrenceville, GA
⁵ Dept. Radiology, Emory University, Atlanta, GA

Elina Sarmah¹, Philip Kennedy²

Neuroscience Journal, Volume 2013, Article ID 594624, 2013.

ABSTRACT

We pursued the question of LFP analysis in the frequency domain. We have discovered that beta peaks defined as being in the 14 to 20 Hz range are present prior or at onset of vocalization. This is similar to motor control studies. These data are still in preparation with confirming studies being completed on intact humans. Here is the abstract of the paper in preparation:

This paper seeks to provide evidence that beta peaks in the frequency spectrum defined as 14 to 20 Hz oscillations in the continuous neural signal were detected at vocalization onset in locked-in, mute patients. The present results were corroborated by EEG recordings in speaking persons. The data were obtained from neural recordings collected from several studies during attempts at inner speech in a mute and paralyzed subject (ER). The present off-line analysis used the continuous neural signal to (1) detect beta peaks meeting the criterion of 0.2% of the power spectrum density (PSD%) or higher, (2) determine the minimum time segments in which the beta peaks could be detected, (3) the strength of the peak and (4) their relationship, if any, to assumed speech onset. To assess the possible functional relatedness of the beta peaks, cross correlation analyses were performed on single unit data before and after beta peaks occurred. Cross correlations were found to increase after the beta peaks. Because the subject was mute, it was not possible to know when speech onset exactly occurred, if at all. Therefore, three speaking subjects were used in a similar testing paradigm but using EEG signals obtained over the speech area. Control studies to rule out EMG contamination effects were also obtained. These studies further suggested that the beta peaks were related to movement of the articulators, not to higher order speech processes.

These results indicate that the beta peak appears to be an indicator of speech onset. This raises the possibility of using these peaks in online applications to assist decoding paradigms being developed to decode speech from neural signal recordings in mute humans.

¹ Georgia Institute of Technology, Atlanta, Georgia, 30313
² Neural Signals Inc., Duluth, Georgia 30096

Philip Kennedy MD PhD

Neurocase 2011 17(5), 381-393

ABSTRACT

An additional paper studied the question of variability of signals when recording from humans and potentially animals. The effect of emotion was noted on our subject and this is described in a single case study paper whose abstract follows:

In many brain areas, modulations in neuronal firing rates are thought to code information.  However, in electrophysiological recording experiments, especially recordings in human patients, the type of information that is coded by a neuron’s discharge patterns is often not known, or difficult to determine. From our long experience with chronic recordings in humans, we have come to suspect that such unexplained modulations in firing rates are often due to state changes in the subject.  We here present two case studies, with extensive data in one subject to illustrate the point that a change in the subject’s emotions, such as sudden fear, surprise, or happiness, may trigger substantial changes in firing rates.

Jess Bartels¹, Dinal Andreasen^1,2, Princewill Ehirim³, Hui Mao⁴, Steven Seibert¹, E Joe Wright¹ and Philip Kennedy¹.

J Neurosci Methods. 2008 Sep 30;174(2):168-76

ABSTRACT

Further more, the following paper describes how to assemble and implant the electrode.

The Neurotrophic Electrode (NE) is designed for longevity and stability of recorded signals.  To achieve this aim it induces neurites to grow through its glass tip, thus anchoring it in neuropil.  The glass tip contains insulated gold wires for recording the activity of the myelinated neurites that grow into the tip.  Neural signals inside the tip are insulated from surrounding neural activity by the glass.  The most recent version of the electrode has four wires inside its tip to maximize the number of discriminable signals recorded from ingrown neurites, and has a miniature connector. Flexible coiled, insulated gold wires connect to electronics on the skull that remain subcutaneous.  The implanted electronics consist of differential amplifiers, FM transmitters, and a sine wave at power up for tuning and calibration.  Inclusion criteria for selecting locked-in subjects include medical stability, normal cognition, and strong caregiver support.  The implant target is localized via an fMRI-naming task.  Final localization at surgery is achieved by 3D stereotaxic localization.  During recording, implanted electronics are powered by magnetic induction across an air gap. Coiled antennas placed on the scalp over the implanted transmitters receive the amplified FM transmitter outputs.  Data is processed as described in the companion paper where stability and longevity issues are addressed.  Five subjects have been successfully implanted with the NE.  Recorded signals persisted for over four years in two subjects who died from underlying illnesses, and continue for over three years in our present subject.

¹ Neural Signals Inc., 3400 McClure Bridge Rd. Building D Suite B, Duluth, GA 30096, USA
² Georgia Institute of Technology, Cobb Co. Atlanta, GA, USA
³ Dept. Neurosurgery, Gwinnett Medical Center, Lawrenceville, GA 30350
⁴ Dept. Radiology, Emory University Hospital, Atlanta, GA 30332

Philip R. Kennedy MD PhD

The Biomedical Engineering Handbook, Third Edition. Ed.: Joe Brazino, 32-1 to 32-14, 2006, revised 2011

ABSTRACT

To help decide which electrode to use, the following updated paper should be helpful:

In the fields of Neural Prosthetics and Neural Engineering there are several viable contenders for the prize of best long-term electrode to access cortical control signals for restoration of communication and movement in humans.  These contenders can be classified into three main groups.  The first group includes those who have developed millimeter-sized tines or pins that are driven into the cortex and provide signals for months and sometimes years [1,11,13,14].  The second group produces flexible wires that are inserted into the cortex and provide signals also for months and sometimes years [2].  The third type of electrode is also a wire configuration but allows for growth of the brain’s neuropil into the hollow glass tip of the electrode that envelops the wires.  Robust signals have been recorded for years from this Neurotrophic Electrode [5,11,13,14]. Thus, these electrodes can be classified into (a) those that protrude towards neurons (tines and wires) and (b) the Neurotrophic Electrode that welcomes the neurites into its tip and thus fuses with the neuropil.