Superintelligence

Paths, Dangers, Strategies

By: Nick Bostrom

Narrated by: Napoleon Ryan

Length: 14 hrs and 17 mins

Publisher's Summary

Superintelligence asks the questions: What happens when machines surpass humans in general intelligence? Will artificial agents save or destroy us? Nick Bostrom lays the foundation for understanding the future of humanity and intelligent life. The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. If machine brains surpassed human brains in general intelligence, then this new superintelligence could become extremely powerful - possibly beyond our control. As the fate of the gorillas now depends more on humans than on the species itself, so would the fate of humankind depend on the actions of the machine superintelligence.

But we have one advantage: We get to make the first move. Will it be possible to construct a seed Artificial Intelligence, to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation?

This profoundly ambitious and original book breaks down a vast track of difficult intellectual terrain. After an utterly engrossing journey that takes us to the frontiers of thinking about the human condition and the future of intelligent life, we find in Nick Bostrom's work nothing less than a reconceptualization of the essential task of our time.

Download the accompanying reference guide.

©2014 Nick Bostrom (P)2014 Audible Inc.

Colossus: The Forbin Project is coming

This book is more frightening than any book you'll ever read. The author makes a great case for what the future holds for us humans. I believe the concepts in "The Singularity is Near" by Ray Kurzweil are mostly spot on, but the one area Kurzweil dismisses prematurely is how the SI (superintelligent advanced artificial intelligence) entity will react to its circumstances.

The book doesn't really dwell much on how the SI will be created. The author mostly assumes a computer algorithm of some kind with perhaps human brain enhancements. If you reject such an SI entity prima facie this book is not for you, since the book mostly deals with assuming such a recursive self aware and self improving entity will be in humanities future.

The author makes some incredibly good points. He mostly hypothesizes that the SI entity will be a singleton and not allow others of its kind to be created independently and will happen on a much faster timeline after certain milestones are fulfilled.

The book points out how hard it is to put safeguards into a procedure to guard against unintended consequences. For example, making 'the greater good for the greatest many' the final goal can lead to unintended consequence such as allowing a Nazi ruled world (he doesn't give that example directly in the book, and I borrow it from Karl Popper who gave it as a refutation for John Stuart Mill's utilitarian philosophy). If the goal is to make us all smile, the SI entity might make brain probes that force us to smile. There is no easy end goal specifiable without unintended consequences.

This kind of thinking within the book is another reason I can recommend the book. As I was listening, I realized that all the ways we try to motivate or control an SI entity to be moral can also be applied to us humans in order to make us moral to. Morality is hard both for us humans and for future SI entities.

There's a movie from the early 70s called "Colossus: The Forbin Project", it really is a template for this book, and I would recommend watching the movie before reading this book.

I just recently listened to the book, "Our Final Invention" by James Barrat. That book covers the same material that is presented in this book. This book is much better even though they overlap very much. The reason why is this author, Nick Bostrom, is a philosopher and knows how to lay out his premises in such a way that the story he is telling is consistent, coherent, and gives a narrative to tie the pieces together (even if the narrative will scare the daylights out of the listener).

This author has really thought about the problems inherent in an SI entity, and this book will be a template for almost all future books on this subject.

It’s a grind

I wasn’t sure how to rate this title, as the actual information found here is the most complete source on the subject matter. However, it is seriously tough to get through. The narrators tone is dull and rather monotone, which is doubly unfortunate as it almost feels like he is reading one long, book sized list of highly technical jargon, specifically designed to lose lamen like myself. This seems like a title it would be better to have a hard copy of, in order to study from, as the combination of the authors writing style and the narrators performance does not make for easy listening.

A must read that must be read slowly

There is not much math in this book, not many pictures or tables. Usually this is a good indicator that I'll be able to follow along in an audio version. That was not true of this book. I listen to audiobooks while doing menial tasks involving infrequent and brief moments of concentration, with most books I am able to do this easily, but this book requires some pondering and digestion. Any distraction seemed to be enough to miss something important. Perhaps some of this was due to narrator's smooth baratone which - for reasons I don't know - I didn't like. I plan on getting the hard copy and reading this one in silence. This book is definitely a must read, but it also seems it must be read slowly. Put it down, think about it, talk about it with your friends, then and only then on to the next chapter.

Book About AI Narrated By AI

I've maintained something of an interest in AI and decided that this book would allow me to go a bit more in-depth. Nope. Whatever degree is required to maintain and understand the analysis that Bostrom puts forward is one that I clearly like. What I mean to say is that Superintelligence is drier than the Sahara and faaaar too long! Worst of all, the narration actually sounds robotic. Bad book, bad narration, bad choice.

Interesting for sure, but kind of boring

Any additional comments?

Every chapter is more or less the author proposing an idea/prediction, and then exhaustively defining and constraining the solution space for that idea. .e.g AI could be done via method X, which would enable A, B, C, D, but would exclude J, K, L, M, N, etc..

Except for that's done over an hour.

So, every detail is treated very well, and it's an interesting process, but near the end I just couldn't take it any more and had to skip parts. :)

Pretty hard to listen to for more than a short tim

Would you say that listening to this book was time well-spent? Why or why not?

I wish it was, but it only takes a couple of minutes before my mind starts wandering and the narrator is just idle background noise.

Would you be willing to try another one of Napoleon Ryan’s performances?

Probably not.

Did Superintelligence inspire you to do anything?

Read the book instead of listen to it.

Any additional comments?

The narrator speaks clearly and eloquently but the tone and meter were just impossible for me to enjoy. He didn't appear to be at all interested or passionate about the subject matter and instead just sounded like he was reading a script full of Star Trek technobabble and was just completely bored.

Lousy and pretentious

If the title sounds great, the author tried too hard to make it a savant book.
He explains at the beginning that it’s not going to be an easy book but what could have been done in much simpler wording, is a soup of useless technical words and long rhetorical sentences.
It’s like instead of saying “yes” to a simple answer you would say: “I agree with the following principles inherent to your question!”
The topic is good but the way it’s written and narrated is just boring, confusing and makes you wonder if there is a start or an end to every chapter...

Thorough, With a Mix of Excellence and Other

The book is worth the listen because it is a very good and thorough exposition of one of the major technological problems and risks approaching us in the very near future. Anything that can bring Popular awareness of this and similar issues is a great value.

On the down side the author is so committed to voicing the scholarly non-committal tone that he fails to make definite statements about any topic, even when he could do so.

At times there are logical fallacies in the arguments, and assumptions about the nature of Artificial Intelligences that appear to be groundless, and are not supported by explanation.

There is also a tendency to quote and rely on a variety of "Celebrity" Experts, who have track records in Technology that more recently have led them down allies of almost clownish obsolescence in one case, and over-confidence leading to fallacies and mistakes in their work in the other case.

I would not take this book as 'gospel' on Super-Intelligence. Rather it is a worthwhile entry into the current fieldwork on the subject, such as it is.

Speculation masquerading as Science

What disappointed you about Superintelligence?

The majority of Nick Bostrom's "SuperIntelligence" is conjecture, and much of it is not credible. The narrative lifts off from a thin crust of scientific fact then tail-spins into unimaginative speculation: some AI projects might advance faster than others; governments would seek to gain control if they thought AI would present a strategic advantage; an AI might become so powerful so fast it could take over the world, resulting in a "singleton" new world order, etc. The book reads like an index of unmoving science fiction premises rather than a thought-provoking expedition over the landscape of possibility.

The author attempts to capture the complex challenges underpinning the development of Artificial Intelligence under the umbrella term "recalcitrance". This leads to absurd simplifications like "Rate of change in intelligence equals optimization power over recalcitrance" on which he bases his theories of an AI "explosion". One of the countless implausible possibilities the author describes is that AI projects could be moved to the cloud, as if they could be scaled as easily as websites.

What do you think your next listen will be?

Emperor of all Maladies

Have you listened to any of Napoleon Ryan’s other performances before? How does this one compare?

SuperIntelligence is rich with phrases like "neurotoxic pollutants" that Napoleon Ryan pronounces with an educated bearing that fits the narrative's imperious tone. The book sounds as if it were written, published and narrated by Orwell's Ministry of Information. The section that describes parental attitudes to improving children's intelligence via genetics is particularly callous.

What character would you cut from Superintelligence?

The author

Any additional comments?

One memorable insight revealed is that pundits who predict the creation of an Artificial Intelligence "within twenty years" are safe in the knowledge that their careers will be over by the time their predictions are proved wrong. We can hope that Bostrom's AI explosion results in robots that can write books more credible, and less soulless, than this.

Bad audio

Audio is surprisingly bad. Seems like the microphone was swallowed!
Story is very good though!

Timely topic, ponderous style and robot narrator

What made the experience of listening to Superintelligence the most enjoyable?

It's a timely topic that is covered in great detail, perhaps stating the obvious a bit too frequently. It is extremely thorough in the approach, delving into a variety of areas that support the central topic of AI and the potential threats it harbours

Who was your favorite character and why?

N/A

What do you think the narrator could have done better?

The narrator is a text to speech synthesiser rather than a real person, and whilst it is better than most, giving a fair degree of intonation, it fails badly at times (eg abbreviations said in a way no human would say) and because the tone is so repetitive any initial advantages in clarity are soon swamped by the monotony.

Did you have an emotional reaction to this book? Did it make you laugh or cry?

No, and this is partly the topic, partly the style and partly the synthetic narrator.

Any additional comments?

It's remarkable that no reviews appear to identify that the narrator is synthetic (one person stated that it was spoken like one). It isn't clear from the sample if you are not prepared but it is obvious once you start to listen, as most words at the same point in a sentence have identical sound, lacking the subtle variation that human narrators provide. It is surprisingly clear but in the long run it becomes extremely tedious to listen to. This also brings the question of whether the book is good value, and given that the narrator being synthetic is not made clear to the purchasers I feel that it is not. Others may disagree and it's a matter of taste but you should be aware before you purchase

Buy the physical book

Who might you have cast as narrator instead of Napoleon Ryan?

Someone directed to do a less hammy and over-dramatic performance of what is a non-fiction book.

Any additional comments?

Had I known the book makes many references to figures in the print version, I wouldn't have downloaded.

The most/last important book you'll ever read

This is an intelligent, passionate and thoughtful book for a general, educated audience. Its hard at times, but saving humanity usually is hard.

I've followed Bostrom's academic writing for sometime on matters relating to existential risk. He's a cogent antidote to conspiracy theories, taking seriously our own and nature's capacity for human extinction.

This book outlines the likelihoods and timescales of different technologies creating an intelligence orders of magnitude beyond our own; the possible outcomes, good and bad, for humanity; and ways we can manage and mitigate the effects. In essence, it's, message is that sooner or later we will likely create an intelligence vastly beyond our own and without careful planning (say, not encoding this intelligence to optimize what we humans care about - freedom of choice, minimizing pain, beauty etc) we could very likely be superseded, if not destroyed.

It's all speculative, of course, as is any book about the future. But it's foolish not to plan for rainy days. This is one of those books that humbles you; makes your daily battle against confectionary or anxiety over relationships or vanity about your position in society seem petty.

Not suitable for listening as an audiobook

This book may be an interesting read, but I felt it was impossible to follow it as an audiobook. Even if the performance was more exciting, the audio lacks the visual content of the book, which is important for understanding.

duff narrator, but gothic victorian accent mellows

couldn't make it past first chapter due to gothic horror narrator and broom cupboard sound

recommended by Elon Musk

This is a no holds bar book on a subject which if given the appropriate explanation on parts which the book expects you to understand would be I feel a few hours longer, so get ready to take notes and Google some Ai terms as there are a few.

I really enjoyed this book even though it has most dystopian potential outlook of the possible future under the wrong Ai model, all I could think was my goodness that's the most terrifying possible place man kind could end up. The people in the know need to I feel take the subject of a Super intelligence more serious than nukeiler war

Great book with thoughtful considerations

I loved the first few chapters, this gave a great outline of where we are up to with the relevant technologies and what the obstacles are for progression. I would recommend buying the book for this even if it consisted of only the first 4-5 chapters.

My only criticism would be that the author fixated a little on the idea of a superintelligence with a very simple goal system e.g. making as many paperclips as possible. My own view is that in the process of recursive self improvement the AI's goal system would develop in line with the rest of it's intellect and it would end up with more sophisticated, not more simplistic goals than humans. This could of course bring it's own risks and is inherently unpredictable, but doesn't necessarily equate to the default of existential catastrophe asserted in the book.

It is of course expected that there will be different views on this and that my own may be wrong.

Doesn't work as an audiobook

I'll be honest and confess I gave up about 1/4 of the way in, the style of the book which is essentially an academic philosophical paper doesn't lend itself to the audiobook format and the reader's leaden montone delivery with frequent mispronunciation doesn't help either.

I came to this thinking it would have some element, understanding and discussion of the science involved but from the little I listened to this doesn't seem to be the case.

The book is a discussion of what might happen with artificial intelligence at some unknown point in the future if we can work out how to bridge vast chasms in science and knowledge and if we don't take appropriate precautions against disaster. While humans have a long track record of failing to consider the long-term consequences of their actions, I think this is an area that we really don't need to worry about for quite a while yet. If we ever do manage to develop something more intelligent than an amoeba maybe I'll revisit this book, but I doubt I'll live that long.

narrator too strange

story is grand like, there are a few figures too that the audio book misses out on. and yeah narrator, pain in the arse. no offence to him, just not my cup of brew.

Deeply Insightful and very thorough. Bad narration

Where does Superintelligence rank among all the audiobooks you’ve listened to so far?

For anyone interested in AI it is a must-read as it covers many possible scenarios that the reader would have never been able to imagine without consulting this book.
However it's not ideal for beginners. Bostrom introduces the concept of superintelligence assuming that the reader is familiar with artificial intelligence, and quickly moves onto scenarios and existential risk.

How could the performance have been better?

The narrator makes no effort to put emotions into what he says. Every sentence, and every statement sound the same no matter what the topic is about. Listening to it is more akin to a text-to-speech narration than a storyteller.Admittedly, the Swedish syntax of short sentences does not help. Nevertheless, narration could be greatly improved.