Neuromorphic Path Planning for Robots in a Disaster Response Scenario

Members: Jorg Conradt, Timmer Horiuchi

Organizers: Jeff Krichmar (University of California, Irvine) Jennifer Hasler (Georgia Institute of Technology)

Disaster Response Scenario

Topic Area

    The topic area will focus on neuromorphic solutions for planning and explore how they might be deployed in real-world settings, such as disaster relief.

    The illustration above depicts a disaster setting that includes a need to find and help humans, as well as various assets, in a timely manner faced with numerous challenges (e.g., clutter, complexity, debris, limited communication, safety hazards for human rescue teams, etc.). In our workgroup we will set up a real-world, mock, disaster response scenario that will include a number of these challenges, which we will address using neuromorphic solutions.

    Projects might fall under three broad categories in which there are open issues both in biology and conventional algorithms, as detailed below. The project challenges will consist of navigating, as well as identifying and assessing mock disaster zones, both indoors and outdoors. Below, we suggest initial first-week projects designed to enable teams to obtain basic skills that will help them pursue more ambitious goals.

Potential Projects

    0 First Week - Basic Skills

    We will start by acquiring some basic skills focused on the following relevant technologies:

  1. Field Programmable Analog Arrays (FPAAs)
  2. Android-Based Robotics (ABR) - (a) object detection and avoidance, (b) simple demonstration of exploration vs. exploitation
  3. Robot Operating System (ROS) based robotics

  4. See ``Background Materials" section below for relevant information to prepare for the workgroup.
    1 Long-Term Path Planning and Navigation

    How do animals and artificial agents (e.g. robots) plan paths (and navigate) through complex environments?

  1. We will explore models utilizing single and multi-resolution maps for path planning based on principles in the entorhinal cortex and the hippocampus
  2. We will implement the multi-resolution maps on FPAA chips implementing Si-based neuron models
  3. We will demonstrate models on programmable hardware (e.g. FPAA) interfaced with Android phones to control a wheeled robotic platform

  4. How do they take into consideration the context of a situation (i.e., situational awareness) and how does this affect planning?

  5. We will explore neurobiologically inspired algorithms in both software and hardware which take context into consideration when making decisions

  6. The figure below illustrates one aspect of this capability, via grid cells in rodents.

    Grid cells in rodents create an internal GPS signal that can be used for navigation
    2 Reactive Planning and Navigation

    How do animals and artificial agents alter their paths in response to rapidly changing environments?

  1. We will explore models that can rapidly respond to movement, novelty, and saliency based on cortical areas MT/MST, and the parietal cortex.
  2. We will implement these algorithms using spiking neural networks (SNNs) that interface with the DAVIS neuromorphic vision sensor.
  3. Models will be demonstrated on Android-based robot or Pioneer robot.

  4. The figure below illustrates one such model that exhibits this capability, via optic flow.

    Video frames streamed from the Android robot Optic flow fields as detected by neurons in V1 and MT;
    the robot steers away from areas with faster optic flow


    We will likely bring the equipment detailed in the table below. If a participant would like to bring their own equipment for use in the projects, please contact organizers to make plans beforehand.
Sensing Processing Actuation
FPAA aVLSI neuron IC
iniLabs DAVIS240
Android Phone
Pioneer P3-DX
MESA Time of Flight Camera
MESA Time of Flight Camera

Invited Participants

Name Institution Site
Will Browne Victoria University of Wellington http://ecs.victoria.ac.nz/Main/WillBrowne
Jorg Conradt Technische Universität München http://www.nst.ei.tum.de/en
Philippe Gaussier Cergy-Pontoise University http://perso-etis.ensea.fr/gaussier/
Scott Koziol Baylor University http://blogs.baylor.edu/scott_koziol/people/
Michael Milford Queensland University of Technology https://wiki.qut.edu.au/display/cyphy/Michael+Milford
Doug Nitz University of California, San Diego http://dnitz.com/#home
Nicolas Oros Brainchip LLC http://www.brainchipinc.com/
John Shepanski Northrop Grumman Corporation N/A

Background Materials

    Virtual Machines (VMs)

  • VirtualBox - VMs for the workshop will be hosted on VirtualBox. Be sure to also download the extension for USB support.