Results from Spike-Based Computation Topic Area
The spike-based computation workgroup examined several issues about computing with spikes in neuromorphic architectures. Several subtopics were discussed in this workgroup.
- What are the advantages of spike representations over conventional image and speech processing representations?
- What is the role of spikes for carrying specific signal information: example temporal derivative information, how do we quantify the information?
- Role of spikes for computation
- Contrasting rate coding vs. timing codes
- How complex a neuron model is required for basic processing tasks?
- How best to incorporate adaptation and learning in spiking networks when solving a task?
- How can more cognitive tasks be programmed in a spiking neuron environment?
GRAND CHALLENGE PROJECT: The main project we executed in this workgroup was building a machine that can play card games. Building this machine is a challenge to see if we can build a system that uses a spike-based neural architecture for computation at different levels and that is able to learn strategies based on observations of its environments. We decided on a hearts card playing system which uses the spike outputs from a webcam and DVS128 retina and the AEREAR2 cochlea for the recognition of cards to be played.
In the project, we processed the output from a spiking hardware sensor (e.g. retina or cochlea) to perform a specific task. In doing these tasks, we would like to address more fundamental issues such as Initially we are simplifying the recognition problems (making a special card deck, fixed card locations, single speaker, etc). We will start with an off-the-shelf cognitive engine to play strategically and remember what cards opponents have used. We are planning a simple speech synthesis engine for the machine to state which card to play from its hand. There were some interesting questions pursued in this project:
- visual card recognition from a spiking imager using simple template matching
- audition card recognition from spiking cochlea output
- Spike based learning, can the system learn to correlate between the auditory and visual representations of cards. Can it generalized? e.g. once it learns the 2 and 3 of diamonds and the 2 of clubs, can it recognize the 3 of clubs without explicitly training for it?
- Attention aspects, how does the system keep track of what is going on, for example, whose turn is it?
- Spike-based cognition, how can the system remember the cards that were played--how can it play strategically
- Can it learn to play better as it plays more?
The complete description and results of the various modules developed in this project are at the following links.
