Criteria for performance evaluation
Journal Title: Judgment and Decision Making - Year 2009, Vol 4, Issue 2
Abstract
Using a cognitive task (mental calculation) and a perceptual-motor task (stylized golf putting), we examined differential proficiency using the CWS index and several other quantitative measures of performance. The CWS index (Weiss & Shanteau, 2003) is a coherence criterion that looks only at internal properties of the data without incorporating an external standard. In Experiment 1, college students (n = 20) carried out 2- and 3-digit addition and multiplication problems under time pressure. In Experiment 2, experienced golfers (n = 12), also college students, putted toward a target from nine different locations. Within each experiment, we analyzed the same responses using different methods. For the arithmetic tasks, accuracy information (mean absolute deviation from the correct answer, MAD) using a coherence criterion was available; for golf, accuracy information using a correspondence criterion (mean deviation from the target, also MAD) was available. We ranked the performances of the participants according to each measure, then compared the orders using Spearman’s rs. For mental calculation, the CWS order correlated moderately (rs =.46) with that of MAD. However, a different coherence criterion, degree of model fit, did not correlate with either CWS or accuracy. For putting, the ranking generated by CWS correlated .68 with that generated by MAD. Consensual answers were also available for both experiments, and the rankings they generated correlated highly with those of MAD. The coherence vs. correspondence distinction did not map well onto criteria for performance evaluation.
Authors and Affiliations
David J. Weiss, Kristin Brennan, Rick Thomas, Alex Kirlik and Sarah M. Miller
Keywords
Related Articles
- EP ID EP677672
- DOI -
- Views 203
- Downloads 0
To increase engagement, offer less: The effect of assortment size on children’s engagement
In a world that offers children abundant activities from which to choose, understanding how to motivate children to engage longer in productive activities is crucial. This paper examines how the offered assortment size a...
Belief in the unstructured interview: The persistence of an illusion
Unstructured interviews are a ubiquitous tool for making screening decisions despite a vast literature suggesting that they have little validity. We sought to establish reasons why people might persist in the illusion th...
Counterfactual thinking and regulatory fit
According to regulatory fit theory (Higgins, 2000), when people make decisions with strategies that sustain their regulatory focus orientation, they ``feel right'' about what they are doing, and this ``feeling-right'' ex...
The representativeness heuristic and the choice of lottery tickets: A field experiment
The representativeness heuristic (RH) has been proposed to be at the root of several types of biases in judgment. In this project, we ask whether the RH is relevant in two kinds of choices in the context of gambling. Spe...
The price of not putting a price on love
We examine financial challenges of purchasing items that are readily-available yet symbolic of loving relationships. Using weddings and funerals as case studies, we find that people indirectly pay to avoid taboo monetary...