I’ve been making progress on my 2014 reading list, already having completed the first five titles. I was hoping to stay disciplined and stick to the list, but a recommendation I got from William Hertling dashed those hopes.  Hertling is the author of The Singularity Series, which is one of my favorite book series (see my reviews here and here). Needless to say, when one of your favorite writers recommends a book, it gets priority and jumps the queue. He suggested I read Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat.

Our Final Invention is a non-fiction work written in the style of a documentary. Barrat interviews a number of the leading minds working on artificial intelligence (AI) in order to chronicle the evolution and future of AI. More importantly, Barrat spends the majority of the book looking at how we might arrive at two distinct AI points. The first is Artificial General Intelligence (AGI), which is when a machine will be as intelligent as a human. The second is Artificial Super Intelligence (ASI), which is the point at which a machine is orders of magnitude more intelligent than a human.

Barrat does not make any predictions on when these points will occur, except to say that it is not a matter of if, but when. His interviews point out that AI techniques and research are advancing at such a rapid rate that it would not be out of the question for AGI, or the singularity, to occur within the next 10-15 years. From there, he discusses a number of scenarios where a leap from AGI to ASI could occur quickly, potentially leaving us at the mercy of the machines.

Barrat’s concern is that we may achieve the goal of ASI without having built in the proper safeguards to create a “friendly” version of ASI, or a way where we can control or stop its development. Without the intrinsic means to protect ourselves from the machines, Barrat raises the concern that we could be slaves, or more comically, pets, for the machines. In the worst case scenario, he points out that machines may consume humans in order to access our matter and molecules to ruthlessly achieve their programmed goals and objectives.

After reading Hertling’s Singularity Series, Our Final Invention is an awakening to the perils of artificial intelligence. In fact, some of the scenarios are so closely linked to Hertling’s trilogy that it felt as though the two worked together on both projects with one big difference. Hertling’s work is fictional, whereas Barrat’s work is not. Barrat is not trying to simply entertain with his book. He is trying to raise awareness regarding the risk of unsupervised development of AI.

While I appreciate his concern, his work triggered deeper thinking about what artificial intelligence is and how we might achieve it. More specifically, I came away with a few questions that have left me baffled and without answers.

1. What is intelligence?
   Today’s version of artificial intelligence is done through brute force. A computer such as Watson is loaded with facts and given access to knowledge in order to defeat its human opponents. A computer wins at chess because it can use brute force to calculate thousands of moves. Computers are used to predict the weather because they can simulate thousands of scenarios based on prior weather patterns. I have a hard time accepting that brute force and knowledge are the same as intelligence. Intelligence also involves matching patterns, evaluating one’s surroundings, and in some cases intuition. Since we don’t understand how human intelligence really works, can we make machines that are more intelligent as humans if both have the same knowledge and are given access to the same information?
2. Can machines be self-aware?
   A large part of being human is being aware of one’s self and our actions. Can we make a computer that sees itself as a being? One that can learn how to evaluate its surroundings and its interactions with others to become better at what it does? Once again, I’m not sure we understand self-awareness and how to transfer that concept into a machine.
3. Can we give machines a conscience?
   Another fundamental element of being human is having a conscience and the ability to discern right from wrong. Do we understand what conscience is well enough to create this trait in a machine? I’m not sure that the trait works the same between different humans. If that is the case, how do we recreate a trait that is not repeatable in humans in a machine? Is it even possible?
4. Can a machine emulate what defines us?
   Humans are a collection of our knowledge and experiences. Somehow, these are molded together to create who we are. Call it what you like – your spirit, your soul. Is it possible for us to create a machine that has a soul? And if so, if we’re able to upload our brain to a machine, does our soul extend into the machine, or does the machine use our conscience to become its own “being”? Yes, it’s an existential question, but it makes me wonder if we can ever build a machine that is in our own likeness, or if we will be able to “live forever” by uploading ourselves to a machine. In fact, after reading the book, I believe that augmentation through implants is a more likely path to superior levels of intelligence rather than machines that can live and think on their own.

I found Our Final Invention to be a thought provoking read. It raised more questions for me than it answered, but it also made me aware that the discussions about how we monitor and control the development of AI need to happen now, not after levels of AGI, and possibly ASI, have been achieved. We need to make decisions on how we, collectively as a human race, are going to implement safeguards to insure that we don’t become a machine’s pet, or worse yet, an input they must consume to achieve their goal.

I’m not planning to put Our Final Invention into my Must Read category of books. However, if you have an interest in AI development, want to understand the conditions and ways it might emerge, and would like to understand the risks of unconstrained AI development, then I would highly recommend the work.

I’d like to thank William Hertling for recommending Our Final Invention to me.