Methodological notes on model comparisons and strategy classification: A falsificationist proposition
Journal Title: Judgment and Decision Making - Year 2011, Vol 6, Issue 8
Abstract
Taking a falsificationist perspective, the present paper identifies two major shortcomings of existing approaches to comparative model evaluations in general and strategy classifications in particular. These are (1) failure to consider systematic error and (2) neglect of global model fit. Using adherence measures to evaluate competing models implicitly makes the unrealistic assumption that the error associated with the model predictions is entirely random. By means of simple schematic examples, we show that failure to discriminate between systematic and random error seriously undermines this approach to model evaluation. Second, approaches that treat random versus systematic error appropriately usually rely on relative model fit to infer which model or strategy most likely generated the data. However, the model comparatively yielding the best fit may still be invalid. We demonstrate that taking for granted the vital requirement that a model by itself should adequately describe the data can easily lead to flawed conclusions. Thus, prior to considering the relative discrepancy of competing models, it is necessary to assess their absolute fit and thus, again, attempt falsification. Finally, the scientific value of model fit is discussed from a broader perspective.
Authors and Affiliations
Morten Moshagen and Benjamin E. Hilbig
Keywords
Related Articles
- EP ID EP677864
- DOI -
- Views 130
- Downloads 0
The power of moral words: Loaded language generates framing effects in the extreme dictator game
Understanding whether preferences are sensitive to the frame has been a major topic of debate in the last decades. For example, several works have explored whether the dictator game in the give frame gives rise to a diff...
Cognitive ability and risk aversion: A systematic review and meta analysis
Are highly intelligent people less risk averse? Over the last two decades scholars have argued the existence of a negative relationship between cognitive ability and risk aversion. Although numerous studies support this,...
It must be awful for them: Perspective and task context affects ratings for health conditions.
When survey respondents rate the quality of life (QoL) associated with a health condition, they must not only evaluate the health condition itself, but must also interpret the meaning of the rating scale in order to assi...
Others’ opinions count, but not all of them: anchoring to ingroup versus outgroup members’ behavior in charitable giving
Because of the large amount of information and the difficulty in selecting an appropriate recipient in the context of charitable giving, people tend to make extensive use of heuristics, which sometimes leads them to wron...
The Regret Elements Scale: Distinguishing the affective and cognitive components of regret
Regret is one of the most common emotions, but researchers generally measure it in an ad-hoc, unvalidated fashion. Three studies outline the construction and validation of the Regret Elements Scale (RES), which distingui...