Decoding Algorithms

Pushing the limits of EEG decoding with new techniques such as DNN (Deep Neural Network) and other non-linear approaches. Can we do a better job of decoding EEG signals and reconstructing or deciding the input sound?

Basic Idea

There are many ways to connect audio and EEG data. We want to investigate a number of different algorithms as they apply to our Neuromorphic problems.

People

Malcolm, Peter, Alain, David, Shih-Chii, Jens, Sahar, Ken, Andrew (hardware), Maarten, Daniel, Emiya

Data

On scratch: - EEG data and sounds from Ed Lalor. PRIVAT until further notice. - cEEGrid (around the ear) data from Maarten: for exploration. - EEG data from Jen: One subject attending to 1) Clean speaker 1, 2) Clean speaker2, 3) Speaker 1 in the mixture, 4) Speaker 2 in the mixture.

Results

We are looking at the following algorithms: Linear Regression (LMS, Vespa, Nima's approach), DNN, CCA and Sahar

Linear Regression

See Lucas' presentation for approaches. And Nima's paper on

Daniel and Maarten have very similar code for this. Available for all.

DNN

Deep Neural Networks, for both regression and correlation

Libraries: Theano and Pylearn2, a Python based deep-learning library, developed and maintained by the Yoshua Bengio Lab (see  http://deeplearning.net/software/theano/ and  http://deeplearning.net/software/pylearn2/)

The code is uploaded as DNNcode.zip. In order to use it you need theano, nose and pylearn2 installed, and before running the "train_fcn.py" to train the network it is necessary to set paths for pylearn2:

export PATH=$PATH:<pylearn2path>/pylearn2/pylearn2/scripts

export PYTHONPATH=$PYTHONPATH:<pylearn2path>

export PYLEARN2_DATA_PATH=<datapath>

The first two paths have to point to the folder where you copied pylearn2 and the second one to the folder where your data is stored (it assumes that the EEG data is in the subfolder /Telluride/EEG/). The updloaded version of the code worked well on the demo data of the four subjects.

Correlation (CCA)

Canonical Correlation Analysis is a way to connect two vector signals with the best rotations....

Attachments