How the public, and scientists, perceive advancement of knowledge from conflicting study results
Journal Title: Judgment and Decision Making - Year 2019, Vol 14, Issue 6
Abstract
Science often advances through disagreement among scientists and the studies they produce. For members of the public, however, conflicting results from scientific studies may trigger a sense of uncertainty that in turn leads to a feeling that nothing new has been learned from those studies. In several scenario studies, participants read about pairs of highly similar scientific studies with results that either agreed or disagreed, and were asked, “When we take the results of these two studies together, do we now know more, less, or the same as we did before about (the study topic)?” We find that over half of participants do not feel that “we know more” as the result of the two new studies when the second study fails to replicate the first. When the two study results strongly conflict (e.g., one finds a positive and the other a negative association between two variables), a non-trivial proportion of participants actually say that “we know less” than we did before. Such a sentiment arguably violates normative principles of statistical and scientific inference positing that new study findings can never reduce our level of knowledge (and that only completely uninformative studies can leave our level of knowledge unchanged). Drawing attention to possible moderating variables, or to sample size considerations, did not influence people’s perceptions of knowledge advancement. Scientist members of the American Academy of Arts and Sciences, when presented with the same scenarios, were less inclined to say that nothing new is learned from conflicting study results.
Authors and Affiliations
Derek J. Koehler and Gordon Pennycook
Keywords
Related Articles
- EP ID EP678448
- DOI -
- Views 174
- Downloads 0
Individuals’ insight into intrapersonal externalities
An intrapersonal externality exists when an individual’s decisions affect the outcomes of her future decisions. It can result in decreasing or increasing average returns to the rate of consumption, as occurs in addiction...
The impact of time limitation: Insights from a queueing experiment
We experimentally explore the effects of time limitation on decision making. Under different time allowance conditions, subjects are presented with a queueing situation and asked to join one of the two given queues. The...
Choosing with confidence: Self-efficacy and preferences for choice
Previous research on the role of choice set size in decision making has focused on decision outcomes and satisfaction. In contrast, little is known about interindividual differences in preferences for larger versus small...
The effectiveness of imperfect weighting in advice taking
We investigate decision-making in the Judge-Advisor-System where one person, the “judge”, wants to estimate the number of a certain entity and is given advice by another person. The question is how to combine the judge’s...
Psychophysics and the judgment of price: Judging complex objects on a non-physical dimension elicits sequential effects like those in perceptual tasks
When participants in psychophysical experiments are asked to estimate or identify stimuli which differ on a single physical dimension, their judgments are influenced by the local experimental context — the item presented...