Above is a picture of a simulated EEG controlled robot (2d rover). The robot is controlled by two signals for desired left-turn and right-turn. These can be entered manually with the buttons, or from a text program file of 0s and 1s. Each desired control class (0 or 1) is used to randomly select an EEG trace (displayed at lower left) from one of two different sets of EEG data (test set #4 shown below) which I recorded previously. The selected 1d vector is then submitted to a time-series classifier which returns a 0 or 1 depending on what class it thinks the vector is from, and that value is then used as the actual control signal to the robot. Each dataset submitted can also be used to train the classifier, if desired.
After about 30-40 training vectors, the classifier produces near-perfect results. In the plot above, the robot is cruising in a slight right turn, producing a large circle. The grid and robot wraps around in X and Y coordinates, producing the overlapping pattern. In future versions of the software, the random EEG selection will be replaced with realtime EEG input, and the simulated robot will be replaced by an actual radio controlled robot.
The process for guiding the robot is as follows:
A program consisting of 25 repetitions of the sequence '0 1 1 0' (100 lines) was used to train the classifier to direct the robot to move in a straight line. At first the robot moves erratically, but by the end of training by 100 vectors (or less), it moves reliably in a straight line:
Running a program
Once the classifier has been trained, the training flag can be turned off and other programs run. Shown below is the result of running a 'figure eight' program which consists of 24 desired left turns followed by 24 desired right turns of 15 degrees each:
The path of the robot can also be displayed from robot coordinates:
While the classifier is correct most of the time, it does occasionally make errors. The long-term path of the robot looping through the 'figure eight' program is therefore not just a repeat of two osculating circles, although the robot does tend to repeat the double-circle path at different places in the grid:
The robot wanders over the grid in the picture above because there is no feedback provided to correct its path, it is simply executing pre-programmed instructions which may occasionally be misclassified by the software. In a real robotic system, realtime feedback to the user will allow for correcting the robot's path as it moves. Feedback can also be applied in the simulation by using the buttons to control the robot, instead of a program.
In this project, I am interested in two modes of robotic operation, which have been simulated above:
Both modes of operation could be evaluated in a maze/race/competition environment, in order to better simulate real-world operation. Two users and robots might compete for a 'prize' in order to provide more motivation for correct and compensative control, both in the electronics/software, and in the human, systems.
Classification of EEG signals is a subset of the classification of general time-series data. The basic idea is that there exist two or more distinct physical processes (such as in the brain) that can each generate examples of time-series data (such as a time-varying effect on an electrical current running through the scalp) so that the signals generated by one process are, in some sense, more like each other and less like the signals generated by another process. If the signals generated by one process are truely different from the signals generated by another process, it should be possible to invent a mathematical analysis, and to program a computer to execute that anaysis, which can tell the difference between signals generated by different processes.
Thus, the computer program can classify datasets by assigning to each a number indicating which process is likely to have generated that dataset. This number can then be used to provide a control signal for a mechanical device such as a 2d robot rover. For example, one class of data might represent a 'turn left' command to the robot, while another class of data represent 'turn right'. If the computer classifier is accurate enough, then the raw, unclassified signals can be used to control the robot via the use of software. In this project, the signals used are the EEG waveforms which are acquired from a single pair of electrodes placed on the scalp (i.e. from 1 channel of data). The underlying processes are the mental states representing a desire to 'turn left' and 'turn right' in some way, although any two externally discernable mental states would serve as well (or possibly better).
Although it will eventually be desirable to use more than one channel of data, and to classify that data into more than two distinct classes (and therefore more than two possible commands to a robot), a single channel of EEG data is sufficient to demonstrate the proof of this concept for two classes.
The classifier software developed in this project is proprietary. However, some features that may distinguish it from classifiers used by other researchers are worth noting:
Several synthetic and real datasets have been used during development of the classifier. Four examples are shown below. In each example, a file of vectors was created which contained data from two different classes, ordered randomly in the file. During each test run, a new random ordering was created for the file, the vectors were then sampled without replacement according to that ordering, and submitted to the classifier one at a time. The correct class of each vector was also used to train the classifier. Each run is different in that it represents a potentially different permutation of the elements in each file.
In the text output below, there are six values reported after each vector has been classified. These values (columns) are:
In the plots below, the data classes are color-coded (blue = 0, red = 1). For each test run, only the last 10 lines of output are shown here. To see the entire output, click on the 'Entire run' links.
Test set 1
This dataset consisted of 100 vectors of sine waves, 50 at 10 Hz, 50 at 12 Hz, all with random phases:
90 94 1 1 0.956 1.000 91 72 0 0 0.956 1.000 92 54 0 0 0.957 1.000 93 90 1 1 0.957 1.000 94 56 1 1 0.957 1.000 95 10 0 0 0.958 1.000 96 17 0 0 0.958 1.000 97 59 0 0 0.959 1.000 98 87 1 1 0.959 1.000 99 77 1 1 0.960 1.000 100 75 1 1 0.960 1.000Entire run
Test set 2
This dataset is similar to #1, but has random noise added to each sinewave:
90 22 0 0 0.956 1.000 91 82 1 1 0.956 1.000 92 5 0 0 0.957 1.000 93 51 1 1 0.957 1.000 94 37 0 0 0.957 1.000 95 65 1 1 0.958 1.000 96 44 1 1 0.958 1.000 97 61 0 0 0.959 1.000 98 21 0 0 0.959 1.000 99 12 0 0 0.960 1.000 100 79 0 0 0.960 1.000Entire run
Test set 3
This dataset consists of simulated EEG data using my cerebral cortex simulator (picture). The two data classes were created by using different parameters for the stimulatory and inhibitory settings of the simulator:
90 53 1 1 0.922 1.000 91 96 1 1 0.923 1.000 92 83 1 1 0.924 1.000 93 98 0 0 0.925 1.000 94 59 0 0 0.926 1.000 95 71 1 1 0.926 1.000 96 67 0 0 0.927 1.000 97 5 0 0 0.928 1.000 98 29 1 1 0.929 1.000 99 89 0 0 0.929 1.000 100 95 1 1 0.930 1.000Entire run
Test set 4
This dataset consists of 60 actual EEG data vectors which were recorded by me previously. Both classes are from occipital electrode configurations (PO3 and PO4):
50 38 0 0 0.940 1.000 51 29 1 1 0.941 1.000 52 42 1 1 0.942 1.000 53 45 0 0 0.943 1.000 54 1 0 0 0.944 1.000 55 43 0 0 0.945 1.000 56 40 1 1 0.946 1.000 57 8 0 0 0.947 1.000 58 12 0 1 0.931 0.950 59 26 0 0 0.932 0.950 60 27 1 1 0.933 0.950Entire run
Here is an archive of the above data (in binary PPC format and text format), a Mac OS X PPC program to read and display it, and a Mac OS X PPC program to convert it to text. Several other data sets in addition to those shown above are included. The text files contain one column for each vector of data. The header shows version(1), dtype(0), #vectors, #pts/vec, rate(Hz), duration(secs), freq0(not applicable to time series data), freqn(N/A). Each column is preceeded by the vector's class(0/1) and a secondary class(-1). Text files are in Unix line format.