l'enfer c'est les autres
This book is more frightening than any book you'll ever read. The author makes a great case for what the future holds for us humans. I believe the concepts in "The Singularity is Near" by Ray Kurzweil are mostly spot on, but the one area Kurzweil dismisses prematurely is how the SI (superintelligent advanced artificial intelligence) entity will react to its circumstances.
The book doesn't really dwell much on how the SI will be created. The author mostly assumes a computer algorithm of some kind with perhaps human brain enhancements. If you reject such an SI entity prima facie this book is not for you, since the book mostly deals with assuming such a recursive self aware and self improving entity will be in humanities future.
The author makes some incredibly good points. He mostly hypothesizes that the SI entity will be a singleton and not allow others of its kind to be created independently and will happen on a much faster timeline after certain milestones are fulfilled.
The book points out how hard it is to put safeguards into a procedure to guard against unintended consequences. For example, making 'the greater good for the greatest many' the final goal can lead to unintended consequence such as allowing a Nazi ruled world (he doesn't give that example directly in the book, and I borrow it from Karl Popper who gave it as a refutation for John Stuart Mill's utilitarian philosophy). If the goal is to make us all smile, the SI entity might make brain probes that force us to smile. There is no easy end goal specifiable without unintended consequences.
This kind of thinking within the book is another reason I can recommend the book. As I was listening, I realized that all the ways we try to motivate or control an SI entity to be moral can also be applied to us humans in order to make us moral to. Morality is hard both for us humans and for future SI entities.
There's a movie from the early 70s called "Colossus: The Forbin Project", it really is a template for this book, and I would recommend watching the movie before reading this book.
I just recently listened to the book, "Our Final Invention" by James Barrat. That book covers the same material that is presented in this book. This book is much better even though they overlap very much. The reason why is this author, Nick Bostrom, is a philosopher and knows how to lay out his premises in such a way that the story he is telling is consistent, coherent, and gives a narrative to tie the pieces together (even if the narrative will scare the daylights out of the listener).
This author has really thought about the problems inherent in an SI entity, and this book will be a template for almost all future books on this subject.
All of the cool parts of all the sciences (and social sciences) are covered in this lecture. The lecture is somewhat equivalent to taking the first year undergraduate course of study where you didn't have to worry about memorizing irrelevant facts or learn the mathematics. He tells you what you need to know about physics, geology, economics, sociology, psychology, and even why deductive systems such as mathematics with it's different orders of infinity is so cool and relevant to understanding the nature of reality.
The lecturer ties each lecture together by linking the growth of each subject by how we first understand the individual item (say a rock), then the relationship between the rocks (say gravity) and then the web for which the rocks live in (say the universe).
The paradigms we use to describe our reality are part of the current understanding and when somebody steps out of that paradigm and sees the world differently we first say they are spouting nonsense, but overtime the new paradigm can take hold. Newton was called crazy (action at a distance, what an absurd concept!), Einstein was challenged until he wasn't then he never accepts the quantum mechanics, and so on.
Always, the lecture educates and entertains. He will tie difficult points to a movie, a book, or a painting and show how it is relevant to the point he is making. "Frankenstein" the book finally makes sense to me.
After reading this book, I feel like I'm a philosopher. Durant really knows how to engage the reader and make him feel smart. He gives enough biography and then summaries for each philosopher covered. The book really profits from the fact it was originally written in 1926 and I get to experience knowing the history that followed. He's such a good writer that the book stands up regardless of the unfolding history.