The Signal and the Noise

Why So Many Predictions Fail - but Some Don't

By: Nate Silver

Narrated by: Mike Chamberlain

Length: 16 hrs and 21 mins

Publisher's summary

Updated for 2020 with a new Preface by Nate Silver.

Nate Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair’s breadth, and became a national sensation as a blogger - all by the time he was 30. He solidified his standing as the nation's foremost political forecaster with his near perfect prediction of the 2012 election. Silver is the founder and editor in chief of the website FiveThirtyEight.

Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.

In keeping with his own aim to seek truth from data, Silver visits the most successful forecasters in a range of areas, from hurricanes to baseball to global pandemics, from the poker table to the stock market, from Capitol Hill to the NBA. He explains and evaluates how these forecasters think and what bonds they share. What lies behind their success? Are they good - or just lucky? What patterns have they unraveled? And are their forecasts really right? He explores unanticipated commonalities and exposes unexpected juxtapositions. And sometimes, it is not so much how good a prediction is in an absolute sense that matters but how good it is relative to the competition. In other cases, prediction is still a very rudimentary - and dangerous - science.

Silver observes that the most accurate forecasters tend to have a superior command of probability, and they tend to be both humble and hardworking. They distinguish the predictable from the unpredictable, and they notice a thousand little details that lead them closer to the truth. Because of their appreciation of probability, they can distinguish the signal from the noise.

With everything from the health of the global economy to our ability to fight terrorism dependent on the quality of our predictions, Nate Silver’s insights are an essential listen.

©2012 Nate Silver (P)2012 Penguin Audio

Critic reviews

"One of the more momentous books of the decade." (The New York Times Book Review)

"Mr. Silver, just 34, is an expert at finding signal in noise.... Lively prose - from energetic to outraged...illustrates his dos and don’ts through a series of interesting essays that examine how predictions are made in fields including chess, baseball, weather forecasting, earthquake analysis and politics...[the] chapter on global warming is one of the most objective and honest analyses I’ve seen...even the noise makes for a good read." (New York Times)

"A serious treatise about the craft of prediction - without academic mathematics - cheerily aimed at lay readers. Silver's coverage is polymathic, ranging from poker and earthquakes to climate change and terrorism." (New York Review of Books)

Learn About Statistics Without All The Math

I had been wanting to read this book since it was released, and I finally had the chance. I wasn't disappointed with it. In this book, the author takes a non-mathematical approach to understanding how statistics enable predictions to be made. Instead of talking about actual formulas or complex theories, he tells stories that give examples of predictions that have failed and those that have been successful. Personally, I would have liked to have been exposed to more of the math; however, I recognize that it would not be for everyone.

The book is divided into 13 chapters, and each chapter aligns with a different example of predictions that are successful and those that are not (more or less). Admittedly, some of the chapters were less interesting to me than others. Nevertheless, I feel as though I was able to learn some very important concepts in each chapter. For example, the author's background is in statistics, and he made a name for himself with baseball statistics. I never thought much about baseball statistics, yet I learned something incredibly valuable from this chapter. The author made a comparison between baseball predictions and presidential election predictions. I would have thought that presidential elections would be far richer in data (because of the magnitude of important); however, that is far, far from the truth. Presidential elections happen only once every four years--and there have been only 57 presidential elections in the entire history of the United States. In contrast, there are more than 57 games of baseball played every single year. The dataset in baseball is insanely rich, and what we learn about predictions in baseball can carry over into other data-rich fields.

Another field that is rich in data is weather prediction. I have never wanted to become a meteorologist in my life. I struggle with getting predictions wrong so often. Even so, this chapter was fascinating because the author describes why it's far easier to predict good weather than it is to predict bad weather. It is those bad weather predictions that seem to go wrong so often that make people question the skills of meteorologists, yet it is statistically less inaccurate than we might think. Another thing that I learned in this chapter is that there is a very, very big difference between meteorology and climatology. Climatology attempts to predict weather patterns over many, many years (e.g. 60 to 100); however, meteorology attempts to predict daily weather patterns. Over the longer duration, it is less challenging to predict weather patterns. It is far more difficult to predict daily fluctuations than it is to predict long-term trends. Incidentally, this heuristic holds true in other fields that the author described in other chapters. The stock market is a perfect example of this. The long-term trends in the stock market are much easier to predict than the daily fluctuations.

Far and beyond, my favorite chapter was the one that covered the game of chess. Chess is my favorite game by far. I am fascinated by the game because, unlike poker (described in a separate chapter of the book), you know everything that your opponent knows. All of the pieces are on the board in front of both players. There are no cards that are being held in your opponent’s hand that you have to guess about. Moreover, you know every possible move that is allowed by both you and your opponent. Even so, with all of that knowledge, people still lose at chess. It seems inconceivable that there were ever be anything other than a draw, yet it happens all the time. Good players win. The author talks a lot about how difficult it is to make predictions about the best play in chess because humans are only able to think about two or three moves at a time. Those players who can think out longer moves seem to do better. Enter the computer. Computers have the ability to calculate more moves in less time than humans. This provides computers with far better predictive power than humans. After Deep Blue defeated Garry Kasparov in 1997, the face of computer chess changed significantly. The author predicts that a human will never again be able to defeat a computer at chess (at that level of competition).

In the end, this last example is what seems to be the framework on which the entire book is built. Technology has changed the way in which we make predictions. Computers have the ability to process more data than ever before. And more data exist than ever before! New fields, like data analytics and big data, are pushing the boundaries of what computers can do with large datasets and their utility in prediction. Of course, some systems (like the weather) are less predictable than others (like chess); however, technology is enabling us to get closer and closer to more precise predictions. The author feels as though this ongoing advance in data and technology will ultimately be helpful in more and more fields include homeland security and the war on terror. I, for one, can't wait to see where it leads us.

Not Freakonomics

I love statistics and data analysis and Nate Silver is very smart and knows a lot about data analysis, yet Nate Silver seems to have an “unhealthy obsession” with data analysis. There are quite a number of interesting bits in The Signal and the Noise but it starts with a long and in depth discussion of baseball statistics, which (if you are apathetic to baseball) is only mildly interesting at best. The overall theme of this book was to show the value of Bayesian statistics. I am a big fan of Bayesian stats, but the descriptions here seem weak and the author sometimes seems to delve into data analysis just for sake of data analysis. It felt a bit like being in the room with an obsessive compulsive geek with a calculator. You may learn a lot, but it might not be a lot of fun.

Hot

Nate Silver is hot right now. As I write this, it is three days before the presidential election and he is predicting an Obama win (82% chance of winning). His insights about stats, opinions, signal and noise are spot on. Although I am still not 100% sure what Bayesian logic is. Overall a great listen full of insight. A note on the narrator. I take back every negative comment I've ever made in my reviews of his performances. He was excellent in this context.

Read this, then read Superforecasting

This is a good book because he focuses more on "why so many predictions fail" then on why "some don't." I went in expecting the author to gloat about his successes with five easy steps for how you can too. It's not like that at all. Instead, it's a sober reflection on all the ways prediction can go wrong, but why real progress is possible if we learn to think about the problem correctly.

Once you read it, go read Superforecasting by Tetlock and Gardner. The two books fit together like it's a sequel. While Silver illustrates all the reasons prediction is hard, Tetlock and Gardner illustrate why certain people—when carefully trained—can become much better at it than your average person. In a sense, they show how to take the lessons from Silver's book and make progress.

Both excellent reads. Can't recommend them enough. We need to understand prediction to evaluate so much of the news coming at us. This is a solid place to start.

A fine subject, another underwhelming book

This book's release coincided roughly with the author's correctly calling the Obama reelection, and I know there was some promotional buzz. I enjoyed the book, but like so many others, it is laden with all sorts of random anecdotes and stories that don't really cohere well, so it serves as a very light repast of actionable, penetrating statistical methods, slathered over with plenty of entertainment and filler. So, it probably nicely fulfills the contemporary book industry formula for popular writing and its marketing.

Great insight, glad I did the audio version!

Would you listen to The Signal and the Noise again? Why?

I would like to listen to parts of it again. There are some sections which had far too much detail for me, but at the same time, that is likely a plus for others.

What was one of the most memorable moments of The Signal and the Noise?

I enjoy the parts about how he comes up with his predictions - which I thought was a bit light in the book. I would have liked more insight on the political polling - but maybe that will be the content for his next book - which I will also buy.

Any additional comments?

It is a tough read and a long book - but it is worth it. If you like predictive math and logic - and want some insight into the future of market research - it is quite valuable.

Rehash

What was most disappointing about Nate Silver’s story?

Mr. Silver rehashes old stories better told by Michael Lewis. The rating agency issue in his first chapter has little to do with Bayesian Theory and more to do with how individuals were compensated. The issue is that the system was designed so that the individual players involved in the system could win when the system itself failed. He unfortunately has the wrong issue for an interesting theorem.

Valuable Information

This book shows how perceptive people have separated relevant data from irrelevant data.

The author is fond of baseball and baseball statistics. Not being fond of baseball, I had to remind myself from time to time that the book is about data and analyzing data and not about baseball. Sometimes the book moves a little slow, but it is full of good information.

This is not a mathematical book about statistics so much as a book about judgment calls and how to evaluate relevance.

The author's view on Bayes's theorem is particularly interesting. I hope he will elaborate on Bayes's theorem with examples in detail in a future book.

Fascinating! Can't get enough of this

What was one of the most memorable moments of The Signal and the Noise?

Reviewing the mortgage crisis and the responsible participants was very interesting, and the fact that a hybrid approach to probabilities (number analysis and expert opinion combined) was superior to the pure Moneyball approach was certainly revealing and practical.I also now appreciate much more the work and efforts of the national weather service, what they're able to accomplish, and why their mission should be important to all of us.

What does Mike Chamberlain bring to the story that you wouldn’t experience if you just read the book?

Heck, I thought it was Nate Silver the whole time. I'm sorry, Mike! I guess that's how authentic it seemed.

Was this a book you wanted to listen to all in one sitting?

I really enjoyed the content and insight it provided, so I burned through it at 3x speed.

Any additional comments?

Really enjoyed it!

Not really anything new.

What did you like best about The Signal and the Noise? What did you like least?

It was clear and generally well presented (accessible to a wide audience). I was well aware that people generally are not good at making predictions, relying too heavily on their heuristics and biases (e.g. failure to use prior probabilities as called for by Bayesian statistics); and the importance of putting "band widths" around probability estimates. I also thought that much too much text was devoted to each of the major topics covered (e.g. weather forecasting, political forecasts, economic forecasts, picking stocks, gambling/poker strategies, etc.). These sections could have been considerably shorter.

What was your reaction to the ending? (No spoilers please!)

"Is that all there is?"

Which character – as performed by Mike Chamberlain – was your favorite?

N/A

Could you see The Signal and the Noise being made into a movie or a TV series? Who should the stars be?

No

Any additional comments?

Generally disappointing, in that it did not expand my knowledge -- though I don't fault the book, as I've read widely in this area.

Great book well narrated

The book came out several years ago and I was worried it was less relevant now or that the material had been covered elsewhere in books that I had read - e.g. Phillip Tetlock’s book. But that is not the case. It remains as fresh and relevant as ever, and has something unique to offer. I had also wondered about Silver’s credentials to comment on all these different types of predictions but he appears to have got on top of the essentials of different disciplines from weather forecasting to earthquake predictions.

If there was one thing that I might have done with less of it was the baseball and other sports betting material but this was still okay.

It also contained one of the clearest explanations of the thesis behind climate change that I’ve heard. However this chapter has been criticised by climate scientists as misleading, as giving too much credence to climate sceptics and cranks, as misrepresenting the IPCC findings and as accusing climate scientists of understanding the uncertainty in forecasts. See https://www.huffpost.com/entry/nate-silver-climate-change_b_1909482

The reading / narration / “performance “ was excellent - fresh, full of expression, witty and fun.

Altogether really worthwhile.

Full of anectodes

This book is complete noise with very little substance. It is boring and not educational at the same time.

Excellent read

Great read and comparable to other similar books, some very good i sites. Highly recommend.

A good follow on from Incerto

If you’ve read Nassim Taleb’s Incerto series this is a good follow on from that (albeit written before at least two of his books we’re written).

If you aren’t aware exactly how bad we are at prediction then listen to this, and also how we get better. Combine it with Taleb’s advice on how to improve predictions and you’ll see the world in a very different way.

Anyone interested in Data Analytics - read this

Data-driven technologies have the power to significantly change our ability to understand and predict the behaviour of systems. They also have the power to lure us into a black hole of wasted effort and frustration.

Through a range of examples and principles, Nate Silver shows how to tell the difference. Should be mandatory reading for anyone involved in data analytics and machine learning.

Well read too. A few words seemed mispronounced, but that might have been a UK-US difference, rather than reading error.

Love this book.

Great book. Well put together and well performed. I’ll buy you both a beer in the improbable event I meet you both 😃

Easy to follow, compelling & educational A*

Thoroughly enjoyed this book. Very interesting with lots of great examples from different disciplines and fields to help make the concepts clear. Statistics and probability was never something I was good at in school (stopping maths at 16!) but this really helped and provided some of the big ideas to be considering, tying together human psychology and principles of maths excellently to better understand how and why models work and why they are better in some circumstances than in others.

Great book that is narrated well.

The book is great for listening, as its not so technical that it referes to graph or equations, but still delivers the problems in layman terms.

Mike does a great narration of the book.