Testing the ability of the surprisingly popular method to predict NFL games
Journal Title: Judgment and Decision Making - Year 2018, Vol 13, Issue 4
Abstract
We consider the recently-developed “surprisingly popular” method for aggregating decisions across a group of people (Prelec, Seung and McCoy, 2017). The method has shown impressive performance in a range of decision-making situations, but typically for situations in which the correct answer is already established. We consider the ability of the surprisingly popular method to make predictions in a situation where the correct answer does not exist at the time people are asked to make decisions. Specifically, we tested its ability to predict the winners of the 256 US National Football League (NFL) games in the 2017–2018 season. Each of these predictions used participants who self-rated as “extremely knowledgeable” about the NFL, drawn from a set of 100 participants recruited through Amazon Mechanical Turk (AMT). We compare the accuracy and calibration of the surprisingly popular method to a variety of alternatives: the mode and confidence-weighted predictions of the expert AMT participants, the individual and aggregated predictions of media experts, and a statistical Elo method based on the performance histories of the NFL teams. Our results are exploratory, and need replication, but we find that the surprisingly popular method outperforms all of these alternatives, and has reasonable calibration properties relating the confidence of its predictions to the accuracy of those predictions.
Authors and Affiliations
Michael D. Lee, Irina Danileiko and Julie Vi
Keywords
Related Articles
- EP ID EP678364
- DOI -
- Views 177
- Downloads 0
Framing effects on bidding behavior in experimental first-price sealed-bid money auctions
Consumers often face prices that are the sum of two components, for example, an online purchase that includes a stated price and shipping costs. In such cases consumer behavior may be influenced by framing, i.e., how the...
Sense of control matters: a long spatial distance leads to a short-term investment preference
Increasingly, we can invest in projects that are distributed around the world through online investment platforms. Will the spatial distance between these projects and ourselves affect our investment preferences? The pre...
A study of fairness judgments in China, Switzerland and Canada: Do culture, being a student, and gender matter?
This study compares judgments of the fairness of economic actions among survey populations in Switzerland, and both student and non-student groups in the People’s Republic of China, with the earlier Kahneman, Knetsch and...
Fear and loathing in Las Vegas: Evidence from blackjack tables
This paper uses proprietary data from a blackjack table in Las Vegas to analyze how the expectation of regret affects peoples’ decisions during gambles. Even among a group of people who choose to participate in a risk-ta...
The role of representation in experience-based choice
Recently it has been observed that different choices can be made about structurally identical risky decisions depending on whether information about outcomes and their probabilities is learned by description or from expe...