2015/nlp15

Neuromorphic Natural Language Processing

Members: arindam basu, Andreas Andreou, Andrew Cassidy, Brandon Carroll, Bruno Umbria Pedroni, Diederik Paul Moeys, Daniel Mendat, David Reverter Valeiras, David Anderson, Eric Hunsberger, Emre Neftci, Giovanny Sanchez-Rivera, Guido Zarrella, John Harris, Paul Isaacs, Jonathan Tapson, Kan Li, Mark Wang, Peter Diehl, Peter Hastings, Philip Tully, Pam White, Ritwik Kulkarni, Rodrigo Alvarez, Sergio Davies, Shih-Chii Liu, shashikant koul, Soumyajit Mandal, Timmer Horiuchi, Yezhou Yang, Zonghua Gu

Organizers: John Harris (UF Gainesville) Christian Huyck (Middlesex University)

The invitees include: • Peter Hastings • Guido Zarrella • Ritwik Kulkarni • Ian Mitchell • Christian Huyck • John Harris

We are studying neuromorphic natural language processing (NLP). This is not speech recognition but how to make sense of the words provided by a speech recognition system or written text from books, papers, email or the web. There has been much work in NLP through the years but we are focusing on neuromorphic solutions aiming at how the brain might solve these kinds of problems.

We are targeting implementations with spiking neurons inspired by the brain. Such constraints are typically ignored by the NLP community. One implementation path we are supporting is the use of PyNN to simulate spiking neural networks on your laptop. One advantage of this path is that it is automatic to compile the resulting code onto special neuromorphic hardware SpiNNaker.

We have divided up the workgroup into three different pieces: PROBLEMS, APPROACHES and IMPLEMENTATIONS.

PRESENTATIONS

A number of neural network simulators have been developed to support scientific research in the field. This presentation categorises them, focusing on the factors which influence the design of such simulators. Indeed, such factors, or properties, compose a trade-off space, where each simulator represents one set of choices. Brian, PyNN and SpiNNaker are analysed as example for each of the defined categories (Software simulators, Programming interfaces and Hardware simulators), providing considerations on the trade-offs that have been set during their design.

PROBLEM GROUPS

The participant will work with speech input and output in realtime using appropriate hardware for the problem, including laptops, smartphones and microcontrollers embedded in robots. Appropriate toolkits and modules will be assembled for students use including , and SpiNNaker.

Sentence/story comprehension:
Automatic summarization or question-answering from a spoken story.
John H, lead
Shashi
Peter H
Chris
Ritwik
Kan
Ian

Word2vec:
Representing word similarities with vector measures.
Guido, lead
Jon T.
Andrew
Philip
Bruno
Eric
Saeed
David

ASR input:
Shih-Chii
Emre

APPROACH GROUPS

Spiking neuron based parsers:
Using spiking networks, possibly Cell Assemblies, to learn finite state automata
Peter H, lead
Kan
Ian
Chris
Ritwik
Philip
Sergio

Recurrent NN (mapping to spiking neurons)
Looking at continuous value, recurrent network, or memory based networks and considering how they can be mapped to spiking neurons.
Jon, lead
Emre
Peter
Philip
Guido
Shih-Chii
Bruno
Eric
Saeed
Sergio

IMPLEMENTATION GROUPS

Ultimately we would like to map our solutions to special purpose, spiking neural hardware for high performance simulations.

PyNN/SpiNNaker:
Sergio, lead

Caffe/TrueNorth?:
Andrew, lead,
Bruno, Emre

Nengo:
Eric

TOOLS

 Python Library for performing speech recognition with the Google Speech Recognition API
 NLTK- Python based Natural Language toolkit
 Kaldi- Open source speech recognition


RESULTS

Results are reported under: https://neuromorphs.net/nm/wiki/2015/Results/nlp


PAPERS

Story Comprehension

Distributed Representation

If you want to train or use word vectors in your work, see the wiki page at  http://neuromorphs.net/nm/wiki/2015/word2vec

Working Memory

code and models

Recurrent Neural Nets

code and models

Parsing

Code Code working on spiNNaker for Cell Assembly and Parser -

Implementation

Attachments