Finite State Machine on True North.

Team: Suraj Honnuraiah, Emre Neftci, Andrew Cassidy, Bruno Umbria Pedroni

Our goal is to understand the computational principles used by the neuronal circuits of the neocortex, and to implement these in cognitive technologies. To this end, we are currently exploring the use of the cutting-edge neuromorphic computing system, IBM TrueNorth (TN), as a platform for testing our models of cortical processing (Figure 1). Previous physiological and anatomical experiments have shown that cortical circuits have stereotypical connection patterns across cortical areas. Examples of prominent circuit motifs are strong recurrent connections between superficial pyramidal cells, and strong inhibition of their action potential generation zones. These two motifs can be combined to form competitive-cooperative circuits, such as the winner-take-all (WTA). Compositions of WTAs, in turn, can be used to form neural state-machines that express simple cognitive properties. In this study, we have developed high-level configuration methods to implement these recurrent neural networks on COMPASS; the simulator of TN (Figure 2,3). The theory of finite-state automaton (FSA) is rich and FSA techniques have been used in a wide range of domains, such as switching theory, pattern matching, pattern recognition, speech processing, hand writing recognition, optical character recognition, encryption algorithm, data compression, indexing and operating system analysis (Petri-net).

In the current work, we derive circuit constraints for implementing FSA on TrueNorth architecture in the context of natural language processing (NLP). Language is a sequential process and we would like to explore the state transducers approach in addressing this problem, these transducers facilitate the description of complex linguistic phenomenon involving morphological alterations and syntactic patterns. Towards this end, we have implemented a simple two state FSA (Figure 4) in Compass and derived state holding (persistent activity) and state transitions condition in the simple context of NLP (Figure 5). The final goal is to implement generic state machines on TN that solves any sequential problem, and an automated mapping technique which allows us to map any state machine on to TN architecture (a Jflap to TN interface). Jflap is a package of graphical tools which can be used as an aid in learning the basic concepts of Formal Languages and Automata Theory (Figure 6).

Research Theme TN

Figure.1 (A) Circuit architecture of cat visual cortex (adapted from Binzegger et.al 2004). (B) Abstract circuit models derived from (A), excitatory neurons and connections are shown in red, Inhibitory neurons and connections are shown in blue. (C) Layout of TrueNorth chip developed by IBM. Simulation of a single neuron in NEURON (D), MATLAB (E) and COMPASS (F) environment, with valid parameters for each case and results the corresponding simulation is shown in (G-I).

Figure.2 (A) MATLAB architecture showing the implementation of 10x10 WTA consisting of excitatory and one global inhibitory neuron. (B) COMPASS architecture showing the implementation of 10x10 WTA consisting of 10 input neurons and excitatory (α) and inhibitory (β1) connection. (C) Transfer function of the COMPASS and MATLAB neuron in recurrent circuit for various self-excitatory and inhibitory feedback weight values. The parameters mapped into COMPASS space by the equation derived in the previous section. For this example, we have chosen input weight (w) = 2 and the self-excitatory and inhibitory feedback is varied in both MATLAB and COMPASS environment.

Figure.3 Non-linear signal amplification (winner selection) in COMPASS and MATLAB. (A) Signal amplification in MATLAB model, the input is shown in black trace and the outputs from 10 neurons are shown for different parametric values in different colours. (B) Signal amplification in COMPASS model, the input is shown in black trace ans the outputs from 10 neurons are shown for different parametric values in different colours. (C) Signal amplification in MATLAB and COMPASS model, the gain for 10 neurons are shown for different parametric values in different colours.

Figure.4 State diagram showing the implementation of a two state automata which toggles between the states depending on the input X. Each state is implemented by a coupled WTA to implement memory and the transitions between them is implemented by a pointer neuron (φ) depending on the input X.State holder pool consists of neurons representing different states (S1 & S2) arranged in WTA configuration, a winning state will be selected based on the pointer weight (φ) and the network parameters (α, β). The state pointer pool consists of the pointer neurons (P12 & P21) arranged in WTA configuration, a winning state will be selected based on the input (X) and the network parameters (α, β).

Figure.5 Output of the two state FSA in Compass. The lower traces in blue and red shows the input spiking activity and the upper traces shows the corresponding state activity. We observe that the state neurons retains their activity even when the inputs are removed demonstrating the work memory of the network with persistent activity and the transitions between states are robust and noise tolerant.

Figure. 6 JFLAP is a software for experimenting with formal languages topics including nondeterministic finite automata, nondeterministic pushdown automata, multi-tape Turing machines, several types of grammars, parsing, and L-systems. In addition to constructing and testing examples for these, JFLAP allows one to experiment with construction proofs from one form to another, such as converting an NFA to a DFA to a minimal state DFA to a regular expression or regular grammar.

Attachments