Event-based real-time sensory motor processing

Members: Asha Gopinathan, Christian Brandli, Christoph Maier, Jorg Conradt, Edmund Lalor, Frederic Broccard, Garrick Orchard, Jongkil Park, Jeffrey Pompe, Kwabena Boahen, Magdalena Kogutowska, Mostafa Rahimi Azghadi, Mathis Richter, Matthew Runchey, Nai Ding, Nils Peters, Michael Pfeiffer, Ryad Benjamin Benosman, Sahar Akram, Shih-Chii Liu, Siddharth Rajaram, Sudarshan Ramenahalli, Thomas Murray, Tobi Delbruck, Theodore Yu, Yulia Sandamirskaya

Organizers: Tobi Delbruck, Christian Brandli, Zurich, anyone else can help organize?


Besides teaching the mechanics of using silicon retinas for event based processing in jAER, this group also will organize a variety of discussions on event-based procesing. Possible subjects are listed below; please add/delete/modify the ideas below. Please see schedule for times for these presentations.

  • Ralph Etienne-Cummings - HMAX models for vision on the IFAT
  • Christian Brandli -  SeeBetter project apsDVS pixels
  • Ryad Benjamin Benosman - Methods for optical flow from the DVS events
  • Tobi Delbruck - Vestibular stabilization and extracting crude fast motion parallax from unstable platforms
  • Kwabena Boahen - Bo Wen's silicon cochlea
  • Shih-Chii Liu - The AER-EAR2 silicon cochlea and applications in spatial audition and speaker ID
  • Jorg Conradt - embedded DVS applications
  • Matthew Runchey - How does that line tracker in the pencil balancer really work?
  • Bernabe Linares-Barranco - Faster, more sensitive DVS pixels
  • Michael Pfeiffer - Gesture tracking with stereo pair of DVS
  • Garrick Orchard - Compressive sensing with events from an ATIS.
  • anyone else, please enter what you would like to discuss here and let Tobi Delbruck know

Methods tutorial

In this workgroup we will work with real time digital signal processing of address-event representation events. Many projects will use [ jAER open source software] and hardware for AER, including using a  silicon retina and  silicon cochlea. We plan to build several robots and develop new algorithms and their implementations.


 jAER (java AER, pronounced jay-er) allows processing the outputs of neuromorphic AER sensors and actuators by binding them with the convenience and low cost of PC computation.

The tutorial introduction will cover the basic software architecture of jAER and show by example how to use jAER with a silicon retina and cochlea and how to develop an event processing method. We plan to present jAER in three sessions and then leave the rest of the learning to the projects where it might be used.

jAER methods tutorial material

  1. See attachment:"ABCs of jAER.pdf" Download for slides introducing event-based jAER processing.
  2. See attachment:"delbruckEventBasedVision_GCOE_Symp_2008.pdf" Download for a short paper about event-based visual processing algorithms and the architecture of jAER.
  3. See jAERSensorsTelluride2012.pdf for an introduction to event-based sensors.
  4. See the javadoc zip files attached below for JOGL (Java OpenGL) and for Java 6 itself. To add this javadoc in netbeans, choose Tools/Libraries, make a new library, name it e.g. JOGL, add the jogl.jar class archive for the classpath, and add the zip archive for the javadoc. For Java 6 documentation, select Tools/Java Platforms and add the java 6 javadoc archive from the attachments.


  1. Goalie robot: We will use a new 6-DOF binocular head developed by the lab of Jorg Conradt with two DVS retinas as the sensor end of a new generation goalie robot.
  2. Vestibular system: We will use a DVS retina mounted together with an electronic rate gyro and acceleromotor to develop a stabilized visual system. In this system, we will try to extract motion parallax which provides powerful clues about 3-d scene structure that can be valuable for obstacle avoidance.
  3. Algorithms: We will spend some of our sessions discussing algorithms for event based processing.

The first version of the robo goalie is shown below

The prototype of the new 6-DOF head is below

From hardware (DVS+cochlea+other)


If you will be doing a project using jAER,

  1.  Sign up for a free sourceforge user account. Please use your real name! Don't forget to complete the registration by clicking on the link in the email that sourceforge sends you.
  2. Send your sourceforge username to Christian Brandli so you can be added as a jAER developer.
  3. Install the Java Development Kit plus the Netbeans Integrated Development Environment (IDE). ({attachment:jdk-7u4-nb-7_1_2-windows-ml.exe Download Windows} {attachment:jdk-6u33-nb-7_1_2-linux-ml.sh Download Linux}). For a different platform, go to  http://netbeans.org for netbeans installers. Choose the Jave SE (standard edition) download.
  4. The full checkout of jAER from sourceforge takes a long time at the workshop because it is about 90MB and takes about 30 minutes at off-peak times. To speed up your install, download the attachment:jaer-3416-working-copy.zip Download of the working copy checkout of jAER from the attachments below.
  5. Unzip it somewhere. You now have a subversion "working copy", but it may not be up to date. You can run jAER by starting the launcher jAERViewer.exe or jAERViewer.sh if you have Java installed.
  6. Install a subversion client if you are running windows (attachment:TortoiseSVN- Download or attachment:TortoiseSVN- Download) or mac OS (Versions). See wiki:2012/svn12?.
  7. You now should be able to quickly update your working copy. Do a subversion update on your working copy of jAER.
  8. You should be able to run jAER from the root of this extract. Try to run jAERViewer.exe (under windows) or jAERViewer.sh from linux or Mac OS X. If you get an error like Class not found, you may also need to install or update your java runtime; jAER needs at least java 1.6.

If you have trouble at some point, please ask Christian Brandli or Tobi Delbruck for help.

Drivers and Setup

Driver locations for the retina and cochlea:


For the cochlea, build and run the project. Select:

USB>Set default firmware for blank device

Click "Choose," navigate to


To get started, once you build and run the project, you must select parameters for biasing and other device configuration. Select


From there select

File>Load Settings

The file select dialog box should be in


  • For the retina, select DVS128Fast.xml or DVS128Slow.xml
  • For the Cochlea, select CochleaAMS1bBoard1.xml


  • Under mac OS, the default choice for the JRE (Java runtime environment) is Java 1.5; you need to change the platform used in netbeans. Ask Tobi Delbruck or 'vaibhav' for help here.
  • JOGL: The Java Open GL libraries used in jAER are part of the jAER checkout. But some of the native code JOGL modules for other platforms may not be up to date. We may need to refresh these to the latest JOGL.
  • See  http://jaer.wiki.sourceforge.net/jAER+installation for more tips about installing jAER. Do not download the jAER package from sourceforge - we are using subversion working copies for development.
  • The most frequent problem is that the native libraries such as those associated with JOGL are not found. The folder where the .dll's (windows), .so's (linux), .jnilib's (macosx) should be listed in order in the JVM environment variable java.library.path. There should only be one definition of java.library.path with all folders in sequence separated by the (system dependent) path separator: ";" (semicolon) under windows, ":" (colon) under linux. For example, under windows:
  • The above definition assumes the startup folder is java, not the root of jaer. The path to jars is then relative to the java startup folder.

Path and file separators

OSPath separatorFile separator
Windows; (semicolon)\ (backslash
MacOSX: (colon)/ (fwdslash
linux: (colon)/ (fwdslash

For background information:

  1. Read the paper Frame-free dynamic digital vision to see how spikes are processing in jAER from the  DVS silicon retina.
  2. Have a look at the  jAER wiki on SourceForge? which has lots of information about jAER.

Using matlab

[ As described on the jAER wiki], you can read logged event data into matlab or generate events in matlab to process with jAER.

You need to add the folder host/matlab from the jAER root to your matlab search path to use the loadaerdat.m and saveaerdat.m functions in matlab. These functions load and save raw address/timestamp data. To extract the actual pixel addresses you can use the functions in the various subfolders.

jAER devloper setup for other than netbeans

(For experts). Your jAER launcher needs to include something like the following set of switches to set up the JNI path and other options. These are normally part of your IDE's project options for launching classes.

-Djava.util.logging.config.file=conf/Logging.properties -Dsun.java2d.opengl=false