Predicting Mental Defects: Does the Model or the Expert Win?
Clinical intuition and test scores as a basis for diagnosis
- Klehr, R.
- Journal of Consulting Psychology, 13, 34-38
- An online version of the paper can be found here
- Want a summary of academic papers with alpha? Check out our free Academic Alpha Database!
The present study has compared the ability of a group of 15 experienced clinicians to diagnose mental defectives and schizophrenics on the basis of purely qualitative interpretation of test responses, with the diagnostic results of applying an objective scoring technique to the same material. The material evaluated consisted of the CVS test responses of 36 subjects: 12 schizophrenics, 12 mental defectives, and 12 normals. The results indicate that the performance of the clinicians under these circumstances is comparable to the use of an objective scoring technique, both performances being significantly better than chance. A control group of graduate students did not differ significantly from chance.
In this experiment 36 subjects were chosen. The subjects were broken into 3 categories:
- Mental defectives
15 advanced level psychologists who had long experience in clinical testing were selected to evaluate these subjects’ “normality.” As a control, 15 inexperienced graduate students were also asked to evaluate the 36 subjects. There are two data sources:
- Results from a standardized intelligence test shown to predict mental defects (CVS)
- Experience and intuition.
The results for the experiment are in Table I. On the left axis we have the actual diagnosis and then we have the performance of the different groups (clinical experts, grad students, and the systematic approach (test)).
How does this work?
Take line 1. As a benchmark, any group should get 4/12 correct identifications because the 3 categories are spread evenly among each group of 12. Of the 12 Schizophrenics, experts (labeled Clin.) called 6.6/12, the system got 6/12. Experts said .7/12 Schiz. were defectives; 4.7/12 were normals. The system said 1/12 were defectives; 5/12 were normal. The students were no better than chance. For calling mental defectives, the experts got 8.8/12 and the machine got 10/12. For normals, experts hit 8.8/12, the machine called 7/12. This study highlights a few aspects of models versus experts:
- Experienced experts and the model have similar performance.
- Inexperienced experts have poor performance.
Note: This site provides no information on our value investing ETFs or our momentum investing ETFs. Please refer to this site.
Join thousands of other readers and subscribe to our blog.
Please remember that past performance is not an indicator of future results. Please read our full disclaimer. The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Alpha Architect, its affiliates or its employees. This material has been provided to you solely for information and educational purposes and does not constitute an offer or solicitation of an offer or any advice or recommendation to purchase any securities or other financial instruments and may not be construed as such. The factual information set forth herein has been obtained or derived from sources believed by the author and Alpha Architect to be reliable but it is not necessarily all-inclusive and is not guaranteed as to its accuracy and is not to be regarded as a representation or warranty, express or implied, as to the information’s accuracy or completeness, nor should the attached information serve as the basis of any investment decision. No part of this material may be reproduced in any form, or referred to in any other publication, without express written permission from Alpha Architect.
Definitions of common statistics used in our analysis are available here (towards the bottom)