Making Neural Nets a Reality

(Examining the Physical Aspects of Neural Networks)

Christopher Satterthwaite

of Quincy, IL

Abstract

Research projects involving neural nets are showing significant progress. They have discovered ways of improving speed by hardwiring the actual network. Furthermore, they have discovered a possible solution for the storage problem, by making the networks dynamic. This article will cover the specifics of the aforementioned discoveries, as well as the reality of neural net’s introduction into society.

Introduction

Neural Nets are currently limited to a relatively small size, for the purpose of efficiency. Any net large enough to hold a sufficient amount of information is going to appear sluggish. This is due to the limitations of the hardware platform currently in use.

Speed Factor

Because digital PC’s store information in unary form (0 or 1), we are forced into a fetch/execute cycle with the processor. The processor must interpret the command, fetch the necessary information from the hard drive (or other storage device), execute the desired computations, and then write the information back to memory. This speed of this process continues to increase, due to newer processors; however, when the speed of the computer network is placed in overall comparison with that of the brain, it’s incredibly slow. In order to attempt replication of our brain, we need a way to increase the speed of our artificial neural network.

This task isn’t as difficult as it may appear. Computer engineers simply need to find a way around the fetch-execute cycle. We need to cut some of the unnecessary time in data transfer. In an attempt to do just that, the idea of creating a hardwired neural network emerged. Such a concept would be possible, if only we could find a way that hardware could both store information and perform calculations. In other words, we needed to construct hardware to function in analog.

Research in Neuromorphics has accomplished this feat. Neuromorphics is an effort to capture the “essence” of biological subsystems. As said by Christof Koch of the California Institute of Technology, neuromorphic engineers are “essentially adapting those features, those algorithms, those tricks that the nervous system came up with through the last 600 million years.” Furthermore, according to Andrew Watson:

“Neuromorphic engineering aims to go much further, by transforming microcircuitry into an analog computing medium resembling neural tissue. ‘If we use that [circuitry] in the peculiar way we do, we can generate physical processes that are similar to neurons,’ says Rodney Douglas, who heads the Institute of Neuroinformatics in Zurich, Switzerland” (1934).

Neuromorphic engineers have constructed a transistor that not only holds a range of charges, making it analog, but is able to perform calculations as well. Carver Mead and his team, from the California Institute of Technology, and Lance Glasser, from the Massachusetts Institute of Technology, designed this floating gate transistor (Watson 1935). This new breakthrough will, no doubt, allow the creation of faster artificial neural nets.

Size Factor

Though the speed gained through the physical enhancement is exceptional, it still fails to solve the size problem. We are still limited to a discrete number of nodes and connections, based on the actual size of the hardware.

Researchers at the Swiss Federal Institute of Technology in Lausanne are presently addressing this dilemma through the use of silicon. Daniel Mange and colleagues have most recently completed an expert system that is self-replicating. It has adapted the same principle that John Von Neumann first envisioned in the 1940’s. The setup is based upon a two dimensional array of identical processors. Each processor, or cell, contains a “random-access memory and a single field programmable gate array.” The latter is a “collection of circuits that can be rewired by software, allowing it to assume new functions.” Also, the two-dimensional arrays each have a “mother cell” that rests at one of the four corners. Each mother cell is programmed with a specific string of bits, recognized as its chromosome, “that encodes all the information necessary for all the cells to function together as a computer” (Taubes 1936).

The overall network is not only able to format itself based upon the mother’s chromosome, but is also able to repair itself by utilizing spare silicon cells outside the grid. If a cell in the current configuration is faulty, the column that it’s in is disabled. A new column is then constructed from the spare outlying cells. The new column is formatted with all the functionality from the old column, and then activated. This self-replicating silicon computer retains an extremely high “fault tolerance,” (Taubes 1936) and perhaps opens the door for a system with growth potential. If it can utilize extra cells to repair itself, perhaps it can also be taught to do the same if it needs additional storage. In other words, with a large number of cells the network may be able to expand if necessary.

AI Acceptance

With the advances in speed and size through the physical platform, the seemingly only other mystery in artificial neural nets is said to lie on the metaphysical level. This mystery involves the mind – our consciousness, awareness, and emotion that comes from our brain, the byproducts of our mental activity. Bruce Hinrichs, a professor of psychology and humanities at Century college in White Bear Lake, Minnesota, has a witty, yet truthful, view on this issue:

“Arcane and illogical philosophical arguments are also often given. These imply that brains are somehow different than other matter in the universe and only brains can experience metaphysical transformations. Although these contentions certainly stimulate entertaining discussions, they are grossly speculative, controversial… and extremely unlikely to be resolved in the near future. So far, a materialist view is completely complementary to today’s wide-ranging multidisciplinary empirical data” (28).

Hinrichs goes on to say that,

“if we are willing to take an open-minded view, then it seems reasonable, even fun, to agree with the experts that a brain is a natural computer, the world’s first computer, the best and brightest computer. There should be nothing threatening nor dehumanizing about this view. In fact, adopting this insight may help us to better understand and accept emotional, behavioral, and cognitive differences between people and the mental problems that beset us” (28).

Conclusion

The discoveries we are making in the AI realm are amazing. They are allowing science to continue in its pursuit of understanding the mechanics behind the brain. The advancements in speed and capacity present a new view for science, but will it be enough in other areas of interest? For example, will philosophers and theologians be able to accept the possibility that the ‘self’ behind the brain may also be replicated? The concepts behind the idea aren’t hard to swallow; however, the connotations of such an idea are not only insulting, but threatening as well.

References

Hinrichs, Bruce. (March/April 1998). Computing the Mind. The Humanist, 26-30.

Taubes, Gary. (26 September 1997). After 50 Years, Self-Replicating Silicon. Science,

Vol. 277, 1936.

Watson, Andrew. (26 September 1997). Why Can’t a Computer Be More Like a Brain?

Science, Vol. 277, 1934-1936.