Computing visual disparity with temporal codes

Abstract

An electronic circuit is presented that computes the visual disparity between to input images. The output of the circuit, combined with the specification of the sensors/stereo camera that provides the images, can be used to find the distance between the sensors and the object photographed. A neuromorphic circuit has been developed to solve the task at hand. The fresh approach of this circuit is to employ temporal coding. The first processing step is therefore to project the analog inputs into the time domain, i.e. into voltage spikes, the latency of which encodes the strength of the input. Thus the further comparison of pixel intensity can be done by asynchronous logic. In theory the chip can compute the right visual disparity given two input images/stimuli. The actual aVLSI implementation has proven to have some limitations, but the chip can still compute the right visual disparity for most static images; after some initial fine-tuning of the biases. (The limitations are due to process variations in the production of the VLSI chip. Ways to reduce the effects of process variation are proposed.) The circuit was implemented as a prototype on an AMS VLSI chip. The circuit takes 128 analog inputs, representing 2 images of 64 pixels. They are delivered as frequency encoded spike trains by a 7-bit AER (address event representation) bus. The output consists of 65 separate spike trains, each representing a disparity, multiplexed in a similar 7-bit AER bus. The frequency of spikes on each individual train represents the probability that the train corresponds to the right disparity.