The Covid-19 pandemic reveals that far from living in an age of incessant technological change, we have been neglecting innovation in exactly the areas where we most need it. Faced with a 17th-century plague, we are left to fall back mainly on the 17th-century response of quarantine and closing the theaters.

Is Innovation Speeding Up or Slowing Down?

It is commonplace today to say that innovation is speeding up, but like much conventional wisdom, it is wrong. Some innovation is speeding up, certainly, but some is slowing down. Take speed itself. In my lifetime of more than sixty years, I have seen little or no improvement in the average speed of travel. Congestion on the roads and at airports has in many cases increased the scheduled travel time between two points. A modern airliner, with its high-bypass engines and less-swept wings, is designed to save fuel by going more slowly than a Boeing 707 did in the 1960s. The record for the fastest manned plane, 4,520 miles an hour, was set by the X-15 rocket plane in 1967 and remains unbroken. Boeing 747s are still flying half a century after they were launched. Concorde, the only supersonic passenger plane, is history.

Moreover, recent decades have seen innovation stalled or rejected in a number of technologies. Nuclear power has been unable to roll out plans for new reactor designs. Genetic modification of crops was effectively rejected by Europe. The flow of new pharmaceutical drugs has slowed to a trickle. Ride-sharing apps have been banned in many cities. As the investor Peter Thiel has pointed out, innovation is now largely a digital phenomenon, because bits are lightly regulated and atoms heavily regulated. On all sides we hear arguments that innovation threatens jobs, the environment, privacy and democracy.

Is the pace of innovation slowing down? What needs to be done to encourage innovators? Join the conversation below.
Of immediate relevance to the current emergency, the development of vaccines has languished in the 21st century as an orphan technology, insufficiently encouraged by governments and ignored by the private sector. New vaccines are rarely profitable. By the time a company develops one for a new epidemic, the worst may be over. Last year Wayne Koff, president of the Human Vaccines Project, warned that the world was poorly prepared for a pandemic because vaccine development “is an expensive, slow and laborious process, costing billions of dollars, taking decades, with less than a 10% rate of success.”

It is not just vaccines. Throughout the economy, with the exception of the digital industry, the West is experiencing an innovation famine. The Austrian economist Joseph Schumpeter’s “perennial gale of creative destruction” has been replaced by the gentle breezes of rent-seeking. Two recent books argue that big companies in cozy cahoots with big government increasingly shy away from change, sheltered against competition by regulation and intellectual property rights. In “The Captured Economy” (2017), Brink Lindsey and Steven M. Teles make the case that to the extent that incomes have been stagnating and opportunities for social mobility drying up, the cause is not too much innovation but too little. In “The Innovation Illusion” (2016), Fredrik Erixon and Bjorn Weigel argue that Western economies have “developed a near obsession with precautions that simply cannot be married to a culture of experimentation.”

Innovation Relies On Freedom.

Innovation relies upon freedom to experiment and try new things, which requires sensible regulation that is permissive, encouraging and quick to give decisions. By far the surest way to rediscover rapid economic growth when the pandemic is over will be to study the regulatory delays and hurdles that have now been hastily swept aside to help innovators in medical devices and therapies, and to see whether such reforms could be applied to other parts of the economy too.

There is nothing new about resistance to innovation. In 1662, William Petty, a pioneering economist, medical professor and land speculator, lamented that “the poor inventor runs the gauntloop of all petulant wits,” with everybody trying to stop him from succeeding. At the time, King Charles II, lobbied by the brewing industry, was trying to ban a novel drink called coffee because it caused fake news. “As for coffee, tea and chocolate,” the king pronounced, “I know no good they do; only the places where they are sold are convenient for persons to meet in, sit half day and discourse with all companies that come in of State matters, talking of news and broaching of lies.”

Hansom cab operators in London furiously denounced the introduction of the umbrella for offering shelter to rain-soaked pedestrians. Margarine was subjected to a decadeslong smear campaign from the dairy industry. Musicians’ unions delayed the playing of recorded music on the radio. The Horse Association of America for many years fought a rear-guard action against the tractor, and the natural-ice harvesting industry frightened people with scares about the safety of refrigerators. You may laugh, but the same happens today: Incumbent vested interests, overcautious regulators, opportunistic activists and rent-seeking patent holders combine to oppose or delay almost every innovation.

Surprisingly, there is no good evidence that patents are helpful, let alone necessary, in encouraging innovation. A 2002 study by Josh Lerner, an economist at Harvard Business School, looked at 177 cases of strengthened patent policy in 60 countries over more than a century, finding that “these policy changes did not spur innovation.” James Watt, Samuel Morse, Guglielmo Marconi, the Wright brothers and many others wasted the best years of their lives in court defending their intellectual property, when they might have been busy developing new devices.

The expiration of patents often results in a burst of innovation, as with 3-D printing, where the recent lapse of three key patents has resulted in notable improvements in quality and a drop in price. The historian Anton Howes, of the Royal Society of Arts in London, points out that the French government bought out Louis Daguerre’s patent for photography in 1839 and made the technology freely available, unleashing a burst of creative innovation. Dr. Howes argues, “As we look to fight coronavirus and any future pandemics, we should perhaps consider which patents—for antivirals, vaccines, ventilators and other hygienic equipment—might be bought out in order to remove…innovation bottlenecks.”

What Is Innovation?

What is innovation and how is it best encouraged? Policy makers tend to assume it is the result of clever people having clever thoughts in ivory towers. But science is just as often the fruit as the seed of technology, and invention is different from innovation. To illustrate the difference, Charles Townes, who won the Nobel Prize for the physics behind the laser in 1964, was fond of telling the story of a beaver and a rabbit looking up at the Hoover Dam. “No, I didn’t build it myself,” says the beaver. “But it’s based on an idea of mine.” Discoverers and inventors often feel they get too little credit or profit from a good idea, forgetting just how much effort was needed to turn that idea into a workable, affordable innovation that actually delivered benefits to people.

Thomas Edison understood better than anybody that trial and error is the key to turning an invention into a useful innovation. In developing the nickel-iron battery, his employees undertook 50,000 experiments. Developing a new technology, he famously said, is 1% inspiration and 99% perspiration. Jeff Bezos makes the same point: “Being wrong might hurt you a bit, but being slow will kill you. If you can increase the number of experiments you try from a hundred to a thousand, you dramatically increase the number of innovations you produce.” It turns out that continuous tinkering to develop and refine a better product is much more important than protecting what you’ve already created.

Far from sudden “eureka” moments, innovation is nearly always a gradual process that proceeds by incremental steps. It is a team sport: There is no individual who can be credited with inventing the computer or the internet. The introduction of short-strawed wheat that revolutionized Asian agriculture in the 1960s, resulting in the defeat of famine, was driven by Manzoor Bajwa in Pakistan and M.S. Swaminathan in India. But they got the idea from Norman Borlaug in Mexico, who got it from Burton Bayles, who got it from Orville Vogel, who got it from Cecil Salmon, who got it from Gonjirô Inazuka in Japan, who got it from…at this point the trail runs cold.

Once started, innovation is so inexorable it looks inevitable: 21 different people had the idea of the lightbulb independently. Many people besides the Google founders came up with search engines in the 1990s. Yet, paradoxically, innovation is impossible to predict. Almost nobody foresaw the usefulness and profitability of search engines. The first half of the 20th century brought dramatic changes in transport, the second half in communication. The switch was not the result of policy: Innovation cannot easily be forced.

We Need Incentives For Innovation, Not Regulation To Restrict It.

Given this unpredictability, politicians should rethink the incentives for innovation. One option is to expand the use of prizes, to replace reliance on grants, subsidies and patents. Britain’s famous Longitude Prize, offered in 1714 for accurately measuring longitude at sea, elicited a solution from an unexpected direction: accurate and robust clocks made by a humble clockmaker, John Harrison. Similar serendipity happens today. One study of the online problem-sharing forum known as Innocentive, where organizations can reward crowdsourced solutions to problems that baffle them, found that “the further the focal problem was from the solvers’ field of expertise, the more likely they were to solve it.” Teflon, Kevlar and the Post-it Note are all examples of useful things developed by people looking for something completely different.

The Nobel Prize-winning economist Michael Kremer came up with a concept called the Advance Market Commitment that would fine-tune prizes as incentives to innovate. In 2007, the Gates Foundation committed $1.5 billion to a prize fund to find a vaccine for Pneumococcus bacteria for use in developing countries, where no pharmaceutical company could make money from it. But rather than just winning a lump sum for meeting the goal, companies were assured that if they succeeded, they would win a contract that paid out for 10 years at a good price. The prize money effectively topped up the sum received by the pharmaceutical firm for every vaccine sold. The result of the auction was three good vaccines, costing $2 per dose, which have been given to 150 million children, saving 700,000 lives.

In June 1983, in the West African country of Burkina Faso, a group of French and Vietnamese scientists together with African colleagues began an experiment. They installed mosquito nets in 36 huts specially equipped with mosquito traps. Half the nets were treated with the insecticide permethrin and half had holes torn in them to simulate the wear and tear of everyday life. For five months, volunteers slept in the huts, while every mosquito that entered or tried to leave the huts was collected, dead or alive. The results showed that the presence of a permethrin-treated net, whether intact or torn, repelled mosquitoes, reducing the number entering the huts by about 70% and increasing the rate at which the insects left the hut from 25% to 97%.

This simple, careful experiment proved to be a breakthrough in the control of malaria in Africa. Supported by the Gates Foundation, impregnated bednets first started to be used on a wide scale in 2003, and that very year malaria mortality began to decline. According to a recent study in the journal Nature, insecticide-treated mosquito nets account for 70% of the six million lives saved world-wide from malaria in recent years, twice as high a percentage as antimalarial drugs and insecticide sprays put together. Globally, the death rate from malaria has halved since 2003.

Dealing with Covid-19 has forcibly reminded governments of the value of innovation. But if we are to get faster vaccines and treatments—and better still, more innovation across all fields in the future—then innovators need to be freed from the shackles that hold them back.

This wsj.com essay is adapted from Mr. Ridley’s new book, “How Innovation Works: And Why It Flourishes in Freedom,” which will be published on May 19 by Harper.