Silicon Neuron Discussion Group

The silicon neuron discussion group met twice and discussed topics about neuromorphic hardware and the effects of noise in computation.

Silicon Neurons In the first meeting, the design of silicon neurons was discussed in sub-65nm processes. There was very little experience in the room at designing neurons at these small feature sizes. It was agreed that the circuits that are commonly used by neuromorphic engineers are only guaranteed to work up to 180nm and that after this point the quality of the capacitors available and the leakage in the transistors becomes very challenging especially for sub-threshold, log-domain and current-mode designs. There was a discussion about why we should/should not move to more modern processes and this seemed to split the designers into two camps: those who design neurons to understand something about neuroscience and/or to demonstrate/build a particular function, and those who aim to build large-scale neuromorphic systems with 1 million+ neurons on a single chip. The "large-scale" camp explained that the need for large-scale systems was due to the need to show that such neurally-inspired systems could compete with commercial systems in terms of computational units. Here the use of smaller feature size technologies will become important in the next few generations of chips. It was agreed that there was no particular need for "small-scale" designers to move to the more challenging technologies although in order to promote the work done in the community it may be something that many designers try over the next few years.

We discussed different processes: EEPROM, SoS, FinFETs, SRAM, and 3D technologies and their pros and cons for designing neuromorphic circuits. There was little consensus on what was the best strategy although it is probably fair to say that the standard CMOS processes are preferred!

Part of the issue of the use or lack of use of small feature size silicon technologies is the cost of obtaining such technology. As cost come down there will be more neuromorphic designs in state-of-the-art processes. Another reason to remain active in using the latest technologies is to try and exert some influence on the silicon foundries. In a wish list to the foundries the group cited: better capacitors and less leakage as being the most important parameters in the current technologies.

Noise In the second meeting we discussed the role of noise in both biology (although somewhat briefly) and in circuits. We decided that there were a number of sources of noise but that these sources are generally common to both biology and silicon. There was very little discussion on whether noise has a role in learning and computation in biology although there seemed to be an assumption that this was the case and perhaps no-one wanted to open that particular can of worms! We discussed the merits of intentionally adding noise into a circuit and cited instances such as dithering in digital circuits and the pseudo-random noise that the SpiNNaker chips must inject into their digital circuits. There was clearly a desire to avoid purposely injecting noise into analogue circuits as most would agree that there is plenty of noise in analogue circuits!

We discussed some examples of circuits that use noise to do computation (Elisabetta added several papers to this page that explain her work with learning chips and noise). There definitely seemed to be a trend in people trying to exploit the mismatch and noise in their circuits. There was some debate about the "redundancies" of Nengo which averages the noise out over many neurons. However, Kwabena indicated that as knowledge increases about how to use noise this may change and efficiency may increase.

Following this there was a side-discussion on the need for better user interfaces that allow neuroscientists etc. to use the hardware that we have build and the need for greater development in this area.

The discussions were insightful and entertaining! There was excellent participation from many of the workshop participants, invitees and organisers. Thanks to everyone for an enjoyable few hours!

The "Tablet" notes are attached below.