40. SOFT-BINARY CODING

Department: Computer Science & Engineering
Faculty Advisor(s): Garrison W. Cottrell

Primary Student
Name: Honghao Shan
Email: hshan@ucsd.edu
Phone: 858-342-8649
Grad Year: 2009

Abstract
Computational models in theoretical neuroscience belong to two categories: functional models derived from computational objectives the neural systems appear to work for, and mechanistic models summarizing the neural mechanisms observed in real neurons. One of the functional models, the sparse coding model, has been used to explain the receptive field properties of V1 simple cells as well as the filtering properties of cochlear nerve fibers. However, like many other functional models, it does not explain how the neural mechanisms observed in real neurons collaborate to achieve the objective function. As a result, it is impossible to use the functional model to enhance our understanding of neurophysiological observations in real neurons, nor to use these observations to verify whether the neural circuit does implement the claimed objective function.

We address this issue by relating the sparse coding model to the recurrent network model, a popular mechanistic model in theoretical neuroscience. We show that mathematically the two models are almost identical, except for two minor discrepancies. One discrepancy is that the sparse coding model imposes an extra constraint on the lateral influence between output neurons, which states that two output neurons should have a strong lateral inhibition if their receptive fields overlap. This extra constraint appears to be supported by recent neurophysiological observations, and allows the recurrent network to produce visual features resembling V1 simple cells' receptive fields when learned from natural image patches. The other discrepancy is that the two models employ different nonlinear activation functions. The activation function in the sparse coding model always serves to suppress the neurons' outputs; however, the recurrent network model usually employs a sigmoid activation function.

We show that the sigmoid activation function used in the recurrent network suggests that the marginal prior used in the sparse coding model should have a peak at zero and two humps on both sides of zero. This is in contrast to the sparse priors normally used in the sparse coding model that are characterized by a peak at zero and two heavy tails on both sides of zero. We name this prior the soft-binary prior because it asks the neural outputs to resemble binary coding while allowing the outputs to take any value instead of being strictly binary. With this prior, the sparse coding model becomes self-stable, produces more biologically realistic visual features, and allows more neurophysiological and psychophysical observations to be studied under the efficient coding framework.

Related Links:

1. www.cs.ucsd.edu/~hshan

« Back to Posters or Search Results