Skip to main content

Incentives, Mock Tests and Weather Forecasts

By
Image: TrinityPrep and Fox
It's exam season, so here's some light, bite-sized economics. By now you have probably laboured your way through piles of frustrating mocks for an alphabet soup of tests. Yet every time you get a less-than-ideal score on one of those Barron's practice tests, you hold onto a sliver of hope: the practice test is generally more difficult than the actual exam.

 This is no coincidence—test prep companies like Barron's likely have economic incentives to design their practice tests questions in a manner such that they are more difficult than what one is likely to face in an actual testing situation. But what are these incentives?

 Strange as it seems, an experiment with weather forecasts in Kansas City could shed some light.

 In April 2007, Kansas City resident J.D. Eggelston's fifth-grade daughter was assigned a school project to keep track of the accuracy of weather forecasts for a week. Intrigued, Eggelston decided to continue his daughter's project—for seven months. From his home, he kept track of forecasts—namely, the chance of precipitation—by the local weather station and compared these forecasts to the results he observed.
Chart: "The Signal and the Noise"
Considering that technology exists to reliably predict rainfall patterns over a period of a few days, the forecasts were almost comically inaccurate, reinforcing the commonly held beliefs that make weather reporters the butt of numerous jokes.

As the graph above indicates, the local meteorologist was far off the mark, with a pronounced bias towards forecasting a much higher probability of rain than the observed frequency. When, for example, he reported with certainty that it would rain the next day, he was wrong one out of three times!

 And it's not that this is just some rogue meteorologist at work in Kansas City. "Rain bias," as it is known, is displayed by well regarded weather forecasters as well. Even The Weather Channel, a forecasting company reputed enough that it is occasionally confused for a government agency, tends to overstate the chance of precipitation. Typically, when there has been a 5% chance of rain the next day, The Weather Channel reports the odds as being four times higher, at 20%.

 Eggelston was baffled by his results, and got in contact with representatives from the weather station. He was surprised by what he learnt. "Presentation takes precedence over accuracy," he was told by one forecaster. The overall message he received was that meteorologists didn't care much for the validity of their forecasts, so long as they did not miss predicting a rainy day. This is because people tend to curse the weatherman who ruins a day at the beach by failing to foresee rain, but they'll take unexpected sun as fortuitous.

 In the weather forecasting industry, there is a strong economic incentive to avoid a false negative, incorrectly predicting that it will not rain, and instead produce a false positive, forecasting a rainy day only for it to be sunny.

With that in mind, let's go back to Barron's.

Generally, when you take a mock test shortly before an exam, you read the mock score as a benchmark level. Ideally, you want your final score to be higher than what you achieved on the practice test, and certainly not lower than it.

 If you scored 740 out of 800 on a practice test and then got a 760 on the actual test, you would come away very satisfied with your effort. Importantly for the test prep company's executives, you might even be inclined to regard your prep book as having challenged you positively and given you extra motivation, leading to a better-than-expected score!

 On the other hand, you would be not at all pleased had you scored 780 on the practice test but managed only 760 when it mattered. Although the score was the same both times, you are immensely dissatisfied with having scored below your target, and you might even—God forbid!—blame that same test prep book for instilling false confidence in you.

 Clearly, from a sales perspective, it makes good business sense for those test prep executives to ensure that you rarely, if ever, score better on their practice tests than you will on exam day.

 So the next time you are disappointed with a mock test score—or are put off by a forecast for a rainy day—take heart, and have a think about the economic incentives that might be at play.

 Good luck with exams—and Happy Mother's Day!

Comments

Post a Comment

Popular posts from this blog

On Myanmar

By
This is adapted from a paper I wrote for my Comparative Politics class at Harvard . A Friendless People Thousands have been killed. Hundreds of thousands have fled. They are a people without a home, much less a state. Such is the plight of the Rohingya, whom a UN spokesperson once called the most friendless people in the world. The United Nations High Commissioner for Human Rights has called the situation a “textbook example of ethnic cleansing” (“UN”). This latest wave of state-sponsored violence follows a series of attacks by militants—a small insurgent group, the Arakan Rohingya Salvation Army (ARSA)—in August on Myanmarese military outposts. The brutal military crackdown is ostensibly targeted at insurgents. Yet, it is unmistakably civilians who are bearing the brunt of the violence, with the UN High Commissioner calling the response “clearly disproportionate” and “without regard for basic principles of international law” (“UN”). The approach of ethnic institutionalism...
Post a Comment
Read more

Harmony in the Middle Kingdom

By
Image: Wikimedia Commons The four decades since Deng Xiaoping spearheaded “Reform and Opening Up” in China have seen growth on a scale unparalleled in modern history. Where hundreds of millions were once destitute, extreme poverty has been all but cast into the annals of history; the world competes to attract the wealth of China’s burgeoning middle and upper classes. Analyses of the country's explosive growth are aplenty. Real GDP has grown 64-fold, literacy is above 95%, and in a once Communist country the private sector, at least on paper, now accounts for 60 percent of economic activity. This piece, however, will look more at the stability of the country—while analysing potential economic factors underpinning this. A number of scholars predicted political and economic instability in China as nation rapidly became prosperous. Yet, in a country transformed, political and economic durability remain. The Chinese Community Party continues to enjoy high levels of popular leg...
Post a Comment
Read more

Where Lurks the Next Crisis?

By
Photo: SCMP Optimism flows freely through the markets today. Asset prices are at record highs: investors are happier than ever to throw money at barely profitable technology companies, debt issuances from dubious companies and volatile governments, and anything blockchain related—from Bitcoin to Ethereum to Insanecoin, which does actually exist.     Most observers think that the coin mania is a bubble waiting to burst. But bears have been calling the whole market a bubble for a while, predicting an end to the years-long economic recovery and stock market boom. The old joke goes that economists have called seven of the last two recessions. It still rings true. Doomsayers have been convinced of a market crash since the post-recession recovery had barely gotten on its legs. But with things as they are today, are we on a path toward crisis?   I will not engage in the perilous business of trying to predict when the next economic downturn or market tumble will occ...
Post a Comment
Read more