Project Introduction

This project concerns the development of simple, low-cost, easy-to-use, portable, EEG hardware and software for the Macintosh computer under the OS X operating system. This technology will be used to provide EEG control of devices such as computer cursors, electrical switches and motors, and gross mechanical devices such as microcontrolled robotic systems, motorized wheelchairs, and other prosthetic equipment. The goal of this project is to create a stand-alone general-purpose neurocontrolled robotic system which does not require the use of an external computer.

The purpose of this project is not to develop 'for profit' products. The purpose of this project is to perform basic scientific and engineering research dedicated toward the acquisition of knowledge and the betterment of human life. The intellectual property developed in this project will be distributed in the public domain at no cost for any non-commercial user. Therefore, the use of this technology for commercial licensing, marketing, or profit is strictly prohibited by the developer.

Use of EEG to Control External Electrical and Mechanical Devices

For nearly the last 100 years it has been a dream, primarily of science fiction writers, to utilize the conscious cognitive activity of the brain to control external mechanical devices. If one assumes that all cognitive mental activity correlates to some corresponding electrical activity of the brain which can be measured outside of the brain, then it is astonishing that this 'dream' has not yet been realized in some form. Given even the current limited understanding of EEG patterns and dynamic processes, it is not too difficult to imagine that this could be accomplished in at least some simplistic form with today's technology.

In order to do this, there is first a need for reliable, repeatable, detection of specific EEG events which are related to internal subjective mental states. One such event might be the occurrence of a specific EEG frequency band such as alpha, located at specific 2d locations on the surface of the head. Events of this type can be readily detected by external sensors. In the case of a single sensor, the distribution of the frequency spectrum of an EEG signal, or the detection of a time-series pattern in that signal, might trigger the action of an external device such as an electrical switch or motor. In the case of multiple sensors, the relative frequency spectra between sensors, or the 2d spatial locus of a particular pattern over time, might be used to provide finer control of a device.

Once this can be performed, events of this type can be used to trigger and control both software 'devices' on a computer (such as cursors, buttons, menus, and other user-interface items which are familiar to computer users), as well as actual physical devices outside of the computer, such as small robots. A canonical test application for this kind of control would be to navigate a small radio-operated rover which is propelled by two bilaterally symmetric motors, with the left and right motors controlled respectively by the relative recognition of EEG signals from different parts of the brain. A more complicated example might involve directing the rover with a 'joystick' simulated in software from the 2d recognition of EEG frequencies or patterns via multiple sensors situated over the surface of the head.

Through long-term use, EEG neurocontrol software could be customized and trained for a particular person to operate more easily and efficiently. Over time, a more detailed lexicon of control symbols could be detected and distinguished from an individual user, allowing finer and more complicated control of software and devices. Eventually the software which has been trained to perform the necessary pattern recognition can be extracted and embedded in an external electronic device via a microcontroller or other field-programmable integrated circuit. The goal of phase III of this project is to create a stand-alone neurocontrol system which enables EEG operation of a motorized wheelchair or other gross physical device, without the use of an intermediate computer system.

Development Phases

I: A Macintosh OS 9 EEG hardware and software prototype has already been completed. This consists of EEG input electronics and display and analysis software which performs spectral analysis via the Fourier transform, signal filtering and other mathematical operations, and 3d color animation of the electrical field of the cortex (see results section below). Completed.

II: Development of a neurocontrol system which can actuate computer interface items such as cursors, menus, and buttons, and external devices such as switches, motors, and microcontrolled robots, using one or more channels of conventional EEG input. A robot simulation program which calculates its control signals from pre-recorded EEG data has already been developed, including an autoclassification algorithm that can sequentially and incrementally train itself on input data in real-time. This software is now being interfaced to an actual, physical, robotic system. In progress.

III: Development of an advanced stand-alone, self-contained, neurocontrol system which does not require a separate computer. This system will be based on industry-standard microcontroller and digital signal processing devices, and will make use of the algorithms and techniques developed during previous phases. This system will be used to create EEG-activated controls for bulk physical devices such as motorized wheelchairs and other prosthetics. This phase will also concentrate on the development of different EEG sensor technology and algorithms to make the neurocontrol system more sensitive to user input and less sensitive to external noise, as well as easier and more convenient to set up and operate in normal environments. In future.

Phase I Results

Phase I of this project has been completed over the past several years. One result of this phase is a Macintosh OS 9 EEG data acquisition and analysis program which has been developed using a custom component software environment. This software can acquire and display one or two channels of EEG data in real-time, and can perform spectral analyses via the Fourier transform to produce cascading frequency plots ('compressed spectral arrays') and to filter data to extract alpha or other frequency band EEG waveforms.


Mac OS 9 EEG software

In addition, a general-purpose Macintosh data acquisition program has been developed that can acquire, display, and analyze 2 channels of signal data, perform spectral analysis via the Fourier transform, filter data, and compare two signals to produce frequency vs. magnitude and frequency vs. phase plots of external electronic hardware. This program was used to develop and test the electronics used to acquire EEG signals during this phase. Part of this software has recently been rewritten for Mac OS X, and it will be used extensively during Phases II-III of the project.


DAQ software for Mac OS 9 and OS X

A standard system of nomenclature termed 'extended 10-20 geometry' exists to specify the positions of electrodes used to acquire multichannel EEG data. During Phase I, an 'electrocap' which contains 32 electrodes fixed in the proper 10-20 positions was used to acquire EEG signals. During Phases II-III of this project, a custom 'hardcap' similar to a bicycling helmet will be developed to provide multichannel EEG input without the tedious electrode application process required by the current electrocap. Recognition of EEG events for control signals is also expected to require only 2-4 channels of input, rather than the 32 (or more) commonly used in clinical practice.


10-20 sensor locations and electrocap

Also during Phase I, a 3d EEG display, analysis, and animation program was developed that can show the results of multichannel EEG acquisition on the surface of the head as a continuous field which changes over time. This program interpolates the time-varying signals at each sensor location in order to draw the shape of a continuous electrical field over the surface of the head, allowing spatial and frequency patterns occurring between sensors to be visualized. A derivative of this technique may be used in Phases II-III to provide 2d control of a robot by detecting and tracking EEG frequencies or patterns as they move across the head.


3d EEG animations

Finally during Phase I, a Fourier filter was developed that can extract the alpha (or other) band EEG frequencies and track their strength and position relative to the entire signal as they wax and wane in time. A derivative of this technique may be used in Phases II-III to detect EEG events using alpha or other frequency bands.


Alpha band extraction


Alpha band 2d animation

Discussion

EEG data acquisition and analysis techniques and technology have not changed much since their invention in the time of Berger and Adrian, almost 100 years ago. Two primary innovations, 1) the use of transistors and solid-state circuits and amplifiers instead of vacuum tubes, and 2) computer storage, display, and analysis of data, have advanced this field of study somewhat, but there has been no change of basic scientific paradigm.

In practice, EEG electrodes are difficult and time-consuming to apply and maintain contact with the subject's scalp, are uncomfortable to the subject, and are susceptible to many debilitating electrophysical phenomena such as AC (60 Hz) and other ambient electrical noise, EMG (musculature), EOG (eye movement), and other artifacts. EEG systems are typically so fragile that for accurate data to be acquired, usually the subject must be immobile and quiescent in an electrically shielded environment during the entirety of the recording session, so that 'real world' applications are unfeasable.

Therefore there is a considerable need for long-term ambulatory EEG acquisition from a single subject outside of the laboratory, in 'normal' environments while the subject is in motion and engaged in habitual day-to-day activities. Although this is marginally possible with current technology, it is rarely performed and the data acquired are degraded by the artifacts described above. Therefore there is a significant need for long-term artifact-free EEG data acquisition while a subject is engaged in normal behavior and function, including acquisition in noisy electrical environments which are encountered in everyday situations.

There is also a need for individualized subject baseline data to be accumulated over a long period of time, rather than the few seconds or minutes which are usually accumulated in the laboratory for clinical purposes. This data must then be correlated with the activities of the subject over the normal course of her/his daily activities, for several days or weeks, in order to form a nominal 'standard electrical picture' of the brain from which to judge any specific variation. Only then can any exceptional electrical activity due to intentional behavior be properly discerned.

Finally, there is a need for repeatable correlation between the acquired EEG numerical data and the subjective experience of different conscious mental states during the course of the day. At present, very little has been accomplished along this direction, due partly to the technical issues mentioned above, but primarily to the general clinical view of the brain as a passive stimulus-response filtering device, rather than as a generator of endogenous volitional behavior. In order to provide the necessary control signals for operating external devices, the electrical activity of the brain must be regarded as a cause, rather than an effect. This operational perspective amounts to a radical paradigm shift in contemporary neuroscience.

This shift can only come about through the development of new EEG hardware and software with which mental events can be clearly and robustly detected, so that 1d and 2d time-series and frequency domain signals may train recognition software on the intention of individual subjects, rather than on any hypothetical, unconscious, and therefore mindless standards.

İSky Coyote 2003-2006