The Dial Is Stuck On The Opinion Polling Machine. It’s Time To Shake It.
There is a scene in the 1979 movie China Syndrome, where Jack Lemmon plays an engineer in the control room of a malfunctioning nuclear power plant. Alarms are blaring, but the engineer is confused because the gauge on the panel shows that there is plenty of water in the reactor to cool the atomic pile. Then he gives the gauge a good hard rap and the indicator suddenly drops to zero. The gauge was stuck, and there is actually no water in the reactor. The glowing core of the reactor is busy melting its way through the earth to China, hence the movie’s name.
The China Syndrome, and the Three Mile Island incident which followed weeks after the movie’s release, effectively killed the nuclear power industry in this country. Or, at least put it on life support. Anti-nuclear activists have been scheming to pull the plug on nuclear power ever since. But the debate on nuclear power is for another time. This article is about indicators and metrics, and how much to trust them.
Before the advent of the electronic display, all gauges and meters used to monitor temperatures, levels, and the like depended on a mechanical device to communicate with us humans. Whether a needle spinning on a dial, or a liquid level in a glass tube, or an ink pen drawing a line on a paper strip, a mechanical linkage of some kind was involved. Mechanical contrivances are all subject to a variety of ills. They can stick, or bend, or get rusty, or wear out. This is why you sometimes see people tapping on gauges that fill the instrument panels of airplanes or ships in old movies. They’re making sure the needle isn’t stuck. A rap would often free up a sticky linkage. Not a foolproof remedy, but a quick and easy check.
An indicator which much attention is paid to these days is the opinion poll. This meter on the dashboard of modern life is intended to measure how people feel about this, that, or the other issue. People post poll questions on-line to ask which shoes look best or where to vacation. Corporations craft marketing strategy based upon the results of polls. And politicians steer policies in large part to reflect what public opinion polls claim to reveal. In short, there’s a lot riding on where this needle points.
While polls aren’t mechanical devices, there are complex mechanisms lurking behind the face of the dial. One doesn’t have to be a pollster to appreciate that an improper sample size, an over-represented subgroup, confusingly-worded questions, or a number of other factors can skew poll results one way or the other. And that’s if the pollster is honestly trying to reflect reality. The higher the stakes, the greater the temptation to make the gauge produce the desired reading.
Of course, pollsters will complain about the mass movement to cell phones, or the trend for people to let it go to voice mail, or even people lying to just get those annoying survey-takers off their phone. All of these undoubtedly complicate accurate polling. But, honestly, a disturbing percentage of poll results nowadays just don’t ring true. They don’t pass the smell test.
The most spectacular example was the polling before the 2016 election. Almost all of it predicted a smashing victory for Hilary Clinton. Which turned out not to be so. In fact, it was shockingly off the mark. Not only did Trump win, but he turned states which were assumed to be deep blue-red, or at least pink. The 2016 election was, in effect, a good hard rap to the gauge of polling. And that meter was found to be firmly stuck pointing left.
Here we are, four years later, and it appears that no one has bothered to service, clean and oil the indicator that is opinion polling. I wonder why.
Actually, I don’t wonder. Polling firms are money making enterprises, and there’s money to be made telling your customers what they want to hear. Or, telling the public what your customers want it to hear.
Another indicator, or cluster of indicators, which needs a good hard rap is the news media. Although, I suspect the networks and newspapers which make up the mainstream media are not so much stuck as completely disconnected, their needles replaced with a needle-shaped line painted on the far left-hand side of the dial. With adjacent red warning lights continually flashing.
This is actually a particularly short-sighted position for pollsters and the media to find themselves in. No matter how much money is made, or how many people are fooled, the only real value any of these organizations have is their credibility. How many wrong predictions do pollsters have to make before they are ignored? How many lies do media outlets have to tell before no one believes them anymore? I have a feeling we’ll find out.
In that scene from the China Syndrome there were other indicators, other instruments, telling the engineer that something was wrong. A separate gauge told him the water was low. There was a disturbing mechanical vibration he could feel and even see in the ripples on the surface of the coffee in his cup. You can see the bewilderment on his face as he struggles to make sense of this puzzle. Then, he realizes what must be happening with horror. That’s what makes the engineer reach out to rap the gauge.
Similarly, in 2016 there were conflicting signs to the official poll results and the lopsided media coverage. They took the form of lawn signs for Trump, and enormous, enthusiastic crowds at Trump rallies. Just ripples on the surface according to the experts and talking heads. Ripples on the surface which implied vigorous churning below. The Democrats chose to ignore the ripples and concentrated instead on the big gauge, which they happen to own. They hoped everyone else would as well. Then came the good, hard rap in November.
Now, it’s 2020, and the same gauges that were pointing left last time seem frozen in place like someone drove a nail into the dial so the needles couldn’t go any further right than Barack Obama. This while cities burn, the Democrats won’t restart the economy, and socialists pull a senile Joe Biden’s strings. Ripples indeed. The media is ignoring ripples so strong that they’re sloshing over the rim of the cup.
It has been my experience that you can only rap a sticky gauge so many times before it’s best to call a technician in to replace the darn thing. Sure, you can just ignore it, since it’s not reflecting reality anyway. But, why have an unreliable instrument taking up room on the panel and confusing the unwary?
This article first appeared at americanthinker.com.
Do we know how economies develop? Obviously not, it seems, or otherwise every country would be doing better than it currently is in these low-growth times. In fact, cases of sustained rapid growth, like Japan beginning in the 1960s, or other Southeast Asian countries a decade later, are so rare that they are often described as “economic miracles.”
Yet when Patrick Collison of software infrastructure company Stripe and Tyler Cowen of George Mason University recently wrote an article in The Atlantic calling for a bold new interdisciplinary “science of progress,” they stirred up a flurry of righteous indignation among academics.
Many pointed to the vast amount of academic and applied research that already addresses what Collison and Cowen propose to include in a new discipline of “Progress Studies.” Today, armies of economists are researching issues such as what explains the location of technology clusters like Silicon Valley, why the Industrial Revolution happened when it did, or why some organizations are much more productive and innovative than others. As the University of Oxford’s Gina Neff recently remarked on Twitter, the Industrial Revolution even gave birth to sociology, or what she called “Progress Studies 1.0.”
This is all true, and yet Collison and Cowen are on to something. Academic researchers clearly find it hard to work together across disciplinary boundaries, despite repeated calls for them to do so more often. This is largely the result of incentives that encourage academics to specialize in ever-narrower areas, so that they can produce the publications that will lead to promotion and professional esteem. The world has problems, as the old saying puts it, but universities have departments. Interdisciplinary research institutes like mine and Neff’s therefore have to consider carefully how best to advance the careers of younger colleagues. The same silo problem arises in government, which is likewise organized by departments.
Moreover, fashions in research can lead to hugely disproportionate intellectual efforts in specific areas. To take one example, the ethics of artificial intelligence is clearly an important subject, but is it really the dominant research challenge today, even in the fields of AI or ethics? The financial incentives embedded in technology companies’ business models seem to me at least as important as morality in explaining these firms’ behavior.
At the same time, some important economic questions are curiously underexplored. For example, in his recent book The Technology Trap, Carl Frey expands on his gloomy view of what automation will mean for the jobs of the future, pointing to the adverse effects that the original Industrial Revolution had on the typical worker. Yet Frey also notes that a later period of automation, the era of mass production in the mid-twentieth century, was one of high employment and increasingly broad-based prosperity. What explains the great difference between those two eras?
Today, the role of research in changing behavior – whether that of government officials or of businesses and citizens – is part of the broader crisis of legitimacy in Western democracies. By the early 2000s, technocrats – and economists in particular – ruled the roost, and governments delegated large swaths of policy to independent expert bodies such as central banks and utility regulators. But then came the 2008 global financial crisis. With real incomes stagnating for many, and “deaths of despair” increasing, it is not surprising that expertise has lost its luster for much of the public.
This leads to a final point about the need for a science of progress: what do we actually mean by “progress”? How should it be measured and monitored, and who experiences it? For many reasons, the standard indicator of real GDP growth, which leaves out much of what people value, will no longer do.
The debate about progress therefore raises profound political and philosophical questions about the kind of societies we want. If the global economy falls into recession, as now seems likely, then social divisions and political polarization will intensify further. And the clear message since the turn of the millennium is that if most people do not experience progress, then society isn’t really progressing at all.
Current academic research – into the impact of new technologies, the economics of innovation, and the quality of management, for example – may be providing ever more pieces of the puzzle. But many crucial questions about economic progress remain unanswered, and others have not yet even been properly posed.