Neural network can be learned quickly by Memristors power


Research team have created the reservoir computing system, this work has taken place in electrical engineering and computer science at the University of Michigan. It’s a new method, the type of neural network, this network called as a reservoir computing system. This is made with memristors could theoretically improve the efficiency of machines to think like human. This has been recently published in Nature Communications.

Reservoir computing systems, which improve on a typical neural network's capacity and reduce the required training time, have been created in the past with larger optical components. However, the U-M group created their system using memristors, which need less space and can be integrated more easily into existing silicon-based electronics.

Memristors are a resistive device that can both perform logic and store data. This contrasts with typical computer systems, where processors perform logic separate from memory modules. In this study, Lu's team used a unique memristor that memorizes events only in the near history.

To train a neural network for a task, a neural network takes in a large set of questions and the answers to those questions. In this process of what's called supervised learning, the connections between nodes are weighted more heavily or lightly to minimize the amount of error in achieving the correct answer.

Lu said, “This requires a recurrent neural network, which incorporates loops within the network that give the network a memory effect. However, training these recurrent neural networks is especially expensive.” He explained, "When transcribing speech to text or translating languages, a word's meaning and even pronunciation will differ depending on the previous syllables."

The team proved the reservoir computing concept using a test of handwriting recognition, a standard benchmark among neural networks. Numerals were broken up into rows of pixels, and fed into the computer with voltages like Morse code, with zero volts for a dark pixel and a little over one volt for a white pixel.

Using only 88 memristors as nodes to identify handwritten versions of numerals, compared to a conventional network that would require thousands of nodes for the task, the reservoir achieved 91 percent accuracy.

To demonstrate this, the team tested a complex function that depended on multiple past results, which is common in engineering fields. The reservoir computing system could model the complex task with minimal error.

Lu plans on exploring two future paths with this research: speech recognition and predictive analysis. "We can make predictions on natural spoken language, so you don't even have to say the full word," Lu said. "We could predict what you plan to say next."