2015/bmd15

Meeting 1 (July 1st 2015)

We recapped last year's discussion of benchmarks.

INE had offered to host datasets, but the new website is not up yet. We should follow up with Giacomo to find out how datasets can be hosted on the website. Although the use dropbox/onedrive/googledrive is possible for small datasets, it becomes impractical for larger amounts of data, such as the 200+GB Tobi has.

Performance metrics beyond accuracy were discussed. Michael Pfeiffer filled us in on discussion in Capo Caccia where 4 main metrics were suggested:
1: Hardware performance in terms of the amount of the rate of computation or the bandwidth that can be achieved (of interest to competing hardware platforms)
2: Resource utilization. This can cover chip/silicon area, power consumption, or computational requirements for an algorithm
3: Accuracy of an algorithm, implementation, or hardware
4: Biological Realism

The idea was raised of encouraging future topic area leaders to include a new grand challenge, benchmark, or dataset along with their proposal. Note was made that a grand challenge at a previous workshop had proved distracting to the workshop and as a result, grand challenges were intentionally avoided at the subsequent year's workshop. Therefore, care should be taken to ensure the challenges are relevant and constructive to the topic area. There may also be topic areas where grand challenges are not suitable.

We agreed to think about how grand challenges can be incorporated in the workshop and discuss further in the next meeting.

Whiteboard from meeting 1/7/2015

Spiking MNIST and Caltech101 datasets

I (Garrick) am working to capture new datasets and make them publicly available. They can be found at  http://www.garrickorchard.com/datasets. For access to the Neuromorphic MNIST (N-MNIST) and Neuromorphic Caltech101 (N-Caltech101) datasets while at the workshop, please download them instead at the scratch at (scratch_letter):\scc15\GarrickSaccadeDatasets?\.

I am also working towards making code publicly available at  http://www.garrickorchard.com/code. At the bottom of this page is a zip file with some basic Matlab functions for filtering data, displaying it, and creating videos.

Attachments