Antarctica Starts Here.

For those of you who watch the tech field, you've no doubt heard of Ray Kurzweil, the inventor, technologist, and futurist who's been promulgating the "The Singularity is near!" meme since the 1980's. Love him or hate him, he's a brilliant man who's invented some fantastic, practical things. One of the things he talks about a great deal is how strong AI, which many now refer to as Artificial General Intelligence (i.e., human-like intelligence and sapience) is just a few years away, and he cites Moore's Law as evidence of this. Of course, a lot of people think he's pushing jetwash and get on with their lives. So, when he blogs about things like machine learning and AI a lot of people are prone to ignore his observations about what is being done right now, mistaking them for hypotheses (which he's also prone to, make no mistake).

So, it came with a groan followed by some surprised Google searches when a post hit one of the Zero State mailing lists about a large scale software simulation of part of a human brain.

At first I thought to myself "Yeah, sure, just like that cat brain simulation back in 2009 was only a very small part of a cat's brain, and not the whole thing." I then did some digging on the project called SPAUN (henceforth capitalized because it's an acronym) and found myself amazed. SPAUN (Semantic Pointer Architecture Unified Network) is a software simulation of a network of 2.5 million human-like neurons, each of which includes neurotransmitter concentrations, synaptic connections, and even voltage thresholds in the models. The project is based out of the University of Waterloo, and they've made the model available for download for people to reproduce their results and hack on if they so choose.
The massive neural network of SPAUN is actually a network of other neural networks which are tailored for specific tasks, such as identifying shapes, gathering data from a simulated eye, positioning a simulated hand, and drawing conclusions based on data it has in its short term (working) memory. In other words, it is capable of perceiving its environment through a 784 pixel "eye", analyzing what its senses take in, deciding what to do, and writing its answer with a simulated hand, arm, and pencil. The experiments they ran are tasks that a psychologist might ask a human child to carry out as part of an IQ assessment. Things like, "Given these three numbers tell me what the next number in the sequence would be," or "Tell me what the next two elements in this pattern would be."

There are three fascinating aspects to the SPAUN experiment that I feel should be emphasized. The first is that between tasks a programmer does not have to go in and rework any of the code. There is no setup time involved in having the neural network run the two tests I described above. The simulation is sufficiently flexible in its capabilities that this is unnecessary. The second is that SPAUN is capable of figuring out which of the tasks that it is to carry out entirely on its own; the researchers don't have to open a terminal window and type (for example), "SPAUN, FIGURE OUT THE FIFTH AND SIXTH ELEMENTS OF THIS PATTERN: 1, 1, 2, 22", they pose the question as-is ("1, 1, 2, 22") and SPAUN does its thing ("3, 333.") without any further interaction with the outside world. The third is that the SPAUN model incorporates simulations of several vastly different areas of the human brain, including the basal ganglia, the visual cortex, the prefrontal cortex, and the thalamus. All in the same model. All running simultaneously. All collaborating with one another, analyzing, passing around, and reanalyzing information in different ways toward different ends.

Oh, and did I mention that SPAUN didn't have to be taught how to do any of those things? The nature of the neural networks in the SPAUN model is that they are inherently capable of carrying out the tasks set before it. It didn't have to be put through its paces by the researchers to train it to perform those tasks before it was given brand-new sets of input to see how it would do. It just... knows how to do them.

The software that SPAUN is implemented in, an open source package called Nengo (Github repository), incorporates a scripting language and a visual development environment that makes it very easy to design very sophisticated neural nets. Nengo was designed with very large scale neural simulations in mind, on the order of hundreds of thousands to millions of neurons. It is unique in that you don't have to train the neural networks you're simulating, which can be a frustrating and error-prone process (though training models are included in the software as well). Instead, Nengo's scripting language (called the Neural Engineering Framework) allows the programmer to define abstract models of what a neural network is supposed to do and the NEF then figures out the proper neuronal configurations to implement that model as a neural network. Due to the fact that Nengo is essentially emulating one kind of processor (neuromorphic) on top of another (commodity von Neumann architecture CPUs), it's slow. It takes a lot of processing power to run the SPAUN model, on the order of dozens of CPUs augmented with the CUDA functionality of lots of high-end graphics cards tied together into a massively parallel virtual computer and a few dozen gigabytes of RAM. As far as SPAUN is concerned, one second that passes is one second. For humans standing outside of the simulation watching it grind away, one second that SPAUN perceives passing is 2.5 hours.

Let's stop and think about that for a minute. For a college or a group of determined hackers, buying the hardware for that kind of processing power would be expensive, to say the least. It should be possible for a determined group of volunteers to hook their computers together in the same way with a delay-tolerant software platform like BOINC, though it might require a significant amount of hacking to make Nengo and BOINC play well together (plus Nengo may not itself be delay-tolerant). It would, however, be entirely feasible to pull together that kind of processing power in Amazon's Elastic Computing Cloud on short notice because they offer high performance computing of the sort SPAUN needs for a scant fraction of the cost of buying the parts and building it yourself. To give you an example, I've seen a bill of $200us from Amazon for a temporary virtual HPC cluster that was used to run a very complex engineering simulation. I did the math on it over lunch later that day, and it would have cost between $15kus and $24kus to buy the hardware to build that kind of cluster ourselves, depending on where we sourced the parts from, plus the time to assemble it, install the software, test, and debug the cluster.

At the rate computer power is a) increasing, and b) getting cheaper, the hardware requirements for neurocomputing simulations like this are going to decline. Faster CPUs are becoming more numerous (my current laptop has 8 CPUs running at 2.20 GHz each, and you can count on that number doubling while the price remains more or less static) and graphics cards are steadily becoming more powerful. GPU-augmented processing is no longer rare and exotic; in fact chances are your computer has that capability right now (the graphics card in my laptop does, which was a happy surprise). What would require dozens of CPUs and graphics cards today might require two thirds of that hardware tomorrow, and even less later on. Projects like Parallella that aim to make massively parallel computing stupidly cheap and easy to use to whomever wants it (tip of the pin to Eugen Leitl for putting that on my radar) are likely to lower the bar to entry to neuroprocessing experimentation even more. So, it seems to me that Kurzweil called this one as well, and I'm patiently waiting for my serving of crow at dinner this evening.

If this at all interests you, I strongly recommend that you watch at least some of the videos generated by SPAUN as it was put through its paces. The videos depict what SPAUN was actually doing while one of the researchers narrates the process in a fairly nontechnical way.

I can't say for sure if this path will lead to true AGI. SPAUN, as it stands now, isn't sapient, nor is it sentient. It's a pretty far cry from HAL or Victor 242, and it's not likely that we're going to have personal copies of a SPAUN-like software system at our beck and call doing things for us to free up the processing power between our ears. SPAUN's not complex enough, and the kind of processing power to run SPAUN is significant enough that very few people would be willing to spend the time and money to assemble it. SPAUN is, however, a solid step in what very well might be the creation of a true AGI - a software application that would be, for all intents and purposes, think, reason, feel, and interact with the outside world like you or I.

Time will most certainly tell.