2013/MemristiveDendrites

Examining how to build dendrites with memristors

Project members: Tara Julia Hamilton, Jonathan Tapson, Greg Cohen, Jens Burger

Background

Memristors are a new emerging nanodevice with properties that are interesting to neuromorphic computation (tunable resistance, non-linearity, threshold, (fading) memory). Memristors have been used in various projects as synaptic weights for VLSI neural networks. These applications exploit the tunable resistance/conductance which serves as the synaptic weights and determines how strong a synaptic connection is. The other mentioned memristor properties are mainly ignored and often even unwanted when used as synaptic weights.

Project Details

SKIM Framework

The SKIM framework is a neural network to detect spike patterns in continuous input streams of varying number of inputs. It has similarities to feed-forward neural networks, but the following fundamental differences: Weights between input and hidden layer are fixed random weights and not trained; only the weights between the hidden and the output layer are trained; instead of having neurons in the hidden layer, dendritic inspired alpha functions (synaptic kernels) are performed that allow a non-linear mapping as well as a memory to capture temporal structures within the data.

Using memristors to build alpha functions

The principle idea of the SKIM architecture is to apply various different alpha functions on the weighted inputs to map the data into a higher dimension for feature extraction. Having different alpha functions is requirement to gain higher dimensionality. Simplified, alpha functions can be described by two time constants. One for the rise, and one for the fall time of the function. Within this project we will try to model these functions by RC circuits. In order to create varying alpha functions we use memristors instead of resistors. That gives us flexible control of the shape of all individual alpha functions.

Within the scope of this project we used the tunable resistance of memristors to determine the shape of the alpha functions. With respect to the memristor properties this means that at this point, like as synaptic weights, we do not use the dynamic change in resistance of the memristors. This functionality can be implemented by memristors that only change their resistance when crossing their threshold voltage.

To utilize circuit level simulation within the SKIM framework the existing Matlab code was rewritten in C++. This was combined with a C++ API for Ngspice. The C++ code was responsible for the initialization of the system and the spike generation. Given the input spikes for each synaptic kernel a SPICE netlist file was created and simulated. The simulation data was used by the C++ to do the linear weight training of the output layer.

Results

The main message of this project is that we can implement alpha functions by extremely simple circuits consisting of two memristors and a capacitor. As mentioned before, the memristors determine the time constants for charging and discharging the capacitor. The capacitor itself does the integration over time and constitutes the memory.

The next figure shows the simulation results when using the memristor/capacitor circuits as alpha functions. On the left one can see the desired output (red), the actual output (blue), and their overlap. The right image relates to the soma potential seen by the output layer. The desired and actual output match each other very well. Furthermore, by adjusting the output neurons threshold better matching could be achieved.

Discussion

This project has proven the overall feasibility of using memrisotrs and capacitors to perform various alpha functions that allow a mapping of input data to a higher dimension for feature extraction on the input data. The presented results were limited by the computational resources required to run large scale C++/SPICE simulations necessary to explore a broader design space.´Furthermore no significant difference in using memristors as fixed resistors or as driving them above their threshold and exploiting their dynamic resistance was observed. To address this issue more complex problems have to be presented to the framework. Another aspect that will be part of future work is to explore different memristor types having different properties and behavior. This could allow for a broader range of possible alpha functions.

Attachments