jAER: software spike-event processing

Members: Anirban Dutta, Andrew Schwartz, Jorg Conradt, Chang-Woo Shin, Daniel B. Fasnacht, Daniel Lofaro, frederic broccard, Paco Gomez-Rodriguez, Guillaume Garreau, John Harris, Ivo Georgiev, Jonathan Dyhr, janet wiles, Kazuki Nakada, Michael Rapson, Qiuyan PENG, Ryad Benjamin Benosman, Raphael Juston, Ravi Shekhar, Sergio Davies, Shih-Chii Liu, Steve Kalik, Spencer Kellis, Cameron Patterson, Tara Julia Hamilton, Trushal Chokshi, Timmer Horiuchi, Tobi Delbruck, Tobias Glasmachers

Organizer: Tobi Delbruck

Learn about the exciting area of real time digital signal processing of address-event representation spike data using [ jAER open source software] and hardware for AER, including using a [ silicon retina].

The open source project  jAER (java AER, pronounced jay-er) unlocks the power of neuromorphic AER sensors and actuators by binding them with the power of PCs.

This tutorial introduction will cover the basic software architecture of jAER and show by example how to use jAER with a silicon retina and cochlea and how to develop an event processing method. Several participants, e.g. Vaibhav Garg, Emre Neftci, and Ryad Benjamin Benosman already have experience either developing in jAER or using it.

Plan

We plan to present jAER in two sessions and then leave the rest of the learning to the projects where it might be used.

Preparation

1. Sign up for the subversion tutorial if you are not experienced using subversion.

The URL you need to use in subversion to browse or check out the jAER repository is  https://jaer.svn.sourceforge.net/svnroot/jaer/trunk .

If you will be doing a project using jAER,

1. The full checkout of jAER from sourceforge takes a long time at the workshop because it is about 90MB and takes about 30 minutes at off-peak times. To speed up your install, download the zip file of the working copy checkout of jAER from the attachments below.
2. Unzip the zip somewhere. You now have a subversion "working copy", but it may not be up to date.
3. Install a subversion client if you are running windows (TortoiseSVN - see attachments below) or mac OS (Versions). See wiki:2010/svn10.
4. You now should be able to quickly update your working copy. Do a subversion update on your working copy of jAER.
5. You should be able to run jAER from the root of this extract. Try to run jAERViewer.exe (under windows) or jAERViewer.sh from linux or Mac OS X. If you get an error like Class not found, you may also need to install or update your java runtime; jAER needs at least java 1.6.
6. Under Windows, you can get the JDK (Java development kit) and the netbeans IDE (integrated development environment) by installing the jdk-netbeans bundle attached below. For a different platform, go to  http://netbeans.org for netbeans installers. Choose the Jave SE (standard edition) download.

If you have trouble at some point, please ask Vaibhav Garg or Raphael Berner or Tobi Delbruck for help. Other participants also have experience: for linux help ask Joshua Auerbach.

Drivers and Setup

Driver locations for the retina and cochlea: ...\jaer\drivers\windows\driverDVS_USBAERmini2\...

For the cochlea, build and run the project. Select: USB>Set default firmware for blank device

Click "Choose," navigate to ...\jaer\DeviceFirmwarePCBLayout\CypressFX2\Firmware_FX2LP_Cochleaams1b\firmwareFX2_Cochleaams1b.bix

To get started, once you build and run the project, select View>Biases

From there select File>Load Settings

The file select dialog box should be in ...\jaer\biasgenSettings

For the retina, select DVS128Fast.xml or DVS128Slow.xml For the Cochlea, select CochleaAMS1bBoard1.xml

Troubleshooting

• Under mac OS, the default choice for the JRE (Java runtime environment) is Java 1.5; you need to change the platform used in netbeans. Ask Tobi Delbruck or Vaibhav Garg for help here.
• JOGL: The Java Open GL libraries used in jAER are part of the jAER checkout. But some of the native code JOGL modules for other platforms may not be up to date. We may need to refresh these to the latest JOGL.
• See  http://jaer.wiki.sourceforge.net/jAER+installation for more tips about installing jAER. Do not download the jAER package from sourceforge - we are using subversion working copies for development.
• The most frequent problem is that the native libraries such as those associated with JOGL are not found. The folder where the .dll's (windows), .so's (linux), .jnilib's (macosx) should be listed in order in the JVM environment variable java.library.path. There should only be one definition of java.library.path with all folders in sequence separated by the (system dependent) path separator: ";" (semicolon) under windows, ":" (colon) under linux. For example, under windows:

  -Djava.library.path=jars;jars\SiLabsNativeWindows
  

• The above definition assumes the startup folder is java, not the root of jaer. The path to jars is then relative to the java startup folder.

Path and file separators

OS	Path separator	File separator
Windows	; (semicolon)	\ (backslash
MacOSX	: (colon)	/ (fwdslash
linux	: (colon)	/ (fwdslash

For background information:

1. Read the paper Frame-free dynamic digital vision to see how spikes are processing in jAER from the  DVS silicon retina.
2. Have a look at the  jAER wiki on SourceForge? which has lots of information about jAER.
3. Sign up for a free account at SourceForge using  Create a SourceForge.net Account and send Tobi Delbruck your SF unix username. This username will be needed to add your as a jAER developer so that you can commit your code.

Using matlab

[ As described on the jAER wiki], you can read logged event data into matlab or generate events in matlab to process with jAER.

You need to add the folder host/matlab from the jAER root to your matlab search path to use the loadaerdat.m and saveaerdat.m functions in matlab. These functions load and save raw address/timestamp data. To extract the actual pixel addresses you can use the functions in the various subfolders.

jAER devloper setup for other than netbeans

(For experts). Your jAER launcher needs to include something like the following set of switches to set up the JNI path and other options. These are normally part of your IDE's project options for launching classes.

-Djava.util.logging.config.file=conf/Logging.properties -Dsun.java2d.opengl=false  
-Dsun.java2d.noddraw=true 
-Djava.library.path=jars;jars\SiLabsNativeWindows
-Djava.security.policy=applet.policy

Projects

The potential projects of this practical tutorial are in topic areas, e.g. wiki:2010/rob10, wiki:2010/spike10, wiki:2010/att10,