Artificial synaptic circuits: Computers that can learn

What?

Neural function in animals depends on the transmission of electrochemical signals across the synapses between nerve cells. The output signal may differ from the input signal, having been shaped and conditioned by various factors, including the strength or ‘weight’ of the synaptic connection itself. Taken in the aggregate, the different short- and long-term weights of all the synapses in a neural system form the basis of memory and adaptive learning.

Unlike the static wires and transistors typically found inside computers, biological neural connections have the capacity to refine and improve their own functional characteristics over time. The specific timing, frequency, and strength of the pulses crossing a synapse can cause the synapse to become more or less transmissive in the future.

The fault tolerance and self-tuning memory capabilities of neural systems are of great interest in computing and nanoelectronics applications. Recent developments in electronics show remarkable sophistication in mimicking the changeability or ‘plasticity’ of biological systems.

Why?

One motivator for the development of synthetic synaptic circuits is that they would likely enhance computer performance.[1] The von Neumann architecture of traditional digital computers is based on sequential processing and the execution of explicit instructions, while for pattern recognition and other ‘fuzzy’ parallel-processing tasks, the analog data processing system of the human brain far outpaces the zeroes and ones of computers.[2]

In fact, the modeling of neuronal behaviour in hardware forms an important basis of study for disciplines such as machine learning, adaptive control, and the brain-machine interface. To date, research to emulate synaptic plasticity and learning has generally relied heavily on numerical simulation, which is computationally expensive and slow.

And especially at the nanoscale, where physical defects begin to rival device structures in size, non-digital devices are intrinsically more tolerant to manufacturing variability and aging.[3]

Who, Where, and When?

The past two decades have seen the birth of many adaptive-learning circuits built from standard digital CMOS technology, capable of modifying their functionality in response to the relative timing of stimuli.[4],[5],[6] Recently, however, Chi-Sang Poon and colleagues at the Massachusetts Institute of Technology, Harvard and the University of Texas Medical School at Houston demonstrated a CMOS device whose plasticity depends not only on the timing of stimuli but also on their rate.[7]

Nor is CMOS the only technology under investigation. Last summer, Masakazu Aono and colleagues at the National Institute of Material Science (NIMS) in Japan and the University of California, Los Angeles, showed similarities to synaptic activity in the electro-ionic behaviour of an electrode made of silver sulfide (AgS2), or common silver tarnish.[8]

And Dominique Vuillaume and colleagues at the University of Lille have emulated synaptic behaviour using a nanoparticle organic memory field effect transistor (NOMFET).[9] Their device demonstrated high tolerance to system variability even under a difference in mean conductivity of a factor of ten.[10]

How?

Each of these technologies exploits the properties of non-biological materials in a unique way. The CMOS building blocks of the MIT ‘iono-neuromorphic’ device are sensitive to ion concentration. In the NIMS AgS2 synapse, a voltage applied across an air gap forms a partial bridge of metal atoms, temporarily raising its conductivity. And in the NOMFET – a thin film of pentacene containing gold nanoparticles – an applied voltage creates holes in the pentacene layer, charging the particles and altering the material’s conductance.[10]

But?

As might be expected, none of these technologies is ready for widespread deployment yet, and each fails to emulate some aspect of organic nervous systems. For example, the MIT device demonstrates spike-timing dependent plasticity (STDP), but it requires programming (obviously not a feature of biological systems). The NIMS device demonstrates rate-dependent plasticity but is limited to short-term plasticity and long-term potentiation. And the NOMFET also demonstrates STDP, but its response speed is hampered by limited charge mobility in the pentacene—nanoparticle channel and a relatively low charge/discharge time constant.

Significant fabrication challenges also remain. The MIT device suffers from the constraints of most CMOS solutions, in that it requires multiple transistors per synapse: 400 in this case.[7] Single-device technologies like the AgS2 electrode and the NOMFET would lend themselves more readily to scaling.

Background
Michel Baudry, Joel L. Davis and Richard F. Thompson (eds.), Advances in Synaptic PlasticityMIT Press (1999)
Pt Jeong and Doo Seok, Resistive Switching in Pt, TiO2, Forschungszentrum Jülich (2008)
John Fulcher and L. C. Jain, Computational Intelligence: A Compendium Front Cover, Springer (2008)

 

References

  1. R. Eckmiller, G. Hartmann and G. Hauske (eds.), Parallel Processing in Neural Systems and Computers, North-Holland, 1990.
  2. R. Eckmiller, G. Hartmann and G. Hauske (eds.), The Computer from Pascal to von Neumann, Princeton University Press, 1980.
  3. M. Pierre, R. Wacquez, X. Jehl, M. Sanquer, M. Vinet and O. Cueto, Single-donor ionization energies in a nanoscale CMOS channel, Nat. Nanotechnol., p. 133–137, 2009.
  4. Han Yang, Bing J. Sheu and Ji-Chien Lee, A nonvolatile analog neural memory using floating-gate MOS transistors, Analog Integr. Circuits Signal Process., p. 19–22, 1992.
  5. O. Fujita and Y. Amemiya, A floating-gate analog memory device for neural networks, IEEE Trans. Electron Devices, p. 2029–2035, 1993.
  6. H. Ishiwara, Proposal of adaptive-learning neuron circuits with ferroelectric analog-memory weights, Japan. J. Appl. Phys., p. 442–446, 1993.
  7. G. Rachmuth, H.Z. Shouval, M.F. Bear and C-S. Poon, A biophysically-based neuromorphic model of spike rate- and timing-dependent plasticity, Proc. Nat'l Acad. Sci., 2011.
  8. Takeo Ohno, Tsuyoshi Hasegawa, Tohru Tsuruoka, Kazuya Terabe, James K. Gimzewski and Masakazu Aono, Short-term plasticity and long-term potentiation mimicked in single inorganic synapses, Nature Mater., p. 591–595, 2011.
  9. Fabien Alibart, Stéphane Pleutin, David Guérin, Christophe Novembre, Stéphane Lenfant, Kamal Lmimouni, Christian Gamrat and Dominique Vuillaume, An organic nanoparticle transistor behaving as a biological spiking synapse, Adv. Funct. Mater., p. 330–337, 2010.
  10. Fabien Alibart, Stéphane Pleutin, Olivier Bichler, Christian Gamrat, Teresa Serrano-Gotarredona, Bernabe Linares-Barranco and Dominique Vuillaume, A memristive nanoparticle/organic hybrid synapstor for neuro-inspired computing, Adv. Funct. Mater., p. 609–616, 2012.

Books