The successes of deep learning have been much celebrated within both academia and industry, with one prominent CEO recently proclaiming that “we’re at the beginning of a new day.”* Is this advance really all that it is cracked up to be? Here I shall attempt to provide a different perspective by examining the historical development of this field, and by considering the structure of biological neural networks and the problems solved by them, which are markedly different than those currently being considered. I shall argue that the hard problems of AI, including aspects of perception, cognition, and motor control, are still just as hard and poorly understood today as they were 50 years ago. The point of this exercise is to draw attention to what is missing, and what more will be needed to make true progress. I shall focus on two components in particular: dynamic routing of information flow, and the use of feedback for inferential computations.
* NVIDIA GTC Taiwan 2017 Keynote address
Bruno Olshausen is a Professor in the Helen Wills Neuroscience Institute, the School of Optometry, and has a below-the-line affiliated appointment in EECS at UC Berkeley. He holds B.S. and M.S. degrees in Electrical Engineering from Stanford University, and a Ph.D. in Computation and Neural Systems from the California Institute of Technology. He did his postdoctoral work in the Department of Psychology at Cornell University and at the Center for Biological and Computational Learning at the Massachusetts Institute of Technology. From 1996-2005 he was on the faculty in the Center for Neuroscience at UC Davis, and in 2005 he moved to UC Berkeley. He also directs the Redwood Center for Theoretical Neuroscience, a multidisciplinary research group focusing on building mathematical and computational models of brain function.
Olshausen’s research focuses on understanding the information processing strategies employed by the visual system for tasks such as object recognition and scene analysis. Computer scientists have long sought to emulate the abilities of the visual system in digital computers, but achieving performance anywhere close to that exhibited by biological vision systems has proven elusive. Dr. Olshausen’s approach is based on studying the response properties of neurons in the brain and attempting to construct mathematical models that can describe what neurons are doing in terms of a functional theory of vision. The aim of this work is not only to advance our understanding of the brain but also to devise new algorithms for image analysis and recognition based on how brains work.