Models vs. Experts #3: Advertising Sales Predictions
A Field Test of Implications of Laboratory Studies of Decision Making
- Ashton, A. H.
- The Accounting Review, 59, 361-375
- A version of the paper can be found here.
- Want a summary of academic papers with alpha? Check out our free Academic Alpha Database!
This study provides real-world evidence about the validity of the results of an experimental lens model study evaluating the prediction accuracy of corporate executives versus that of regression models. The decision task involves predictions of annual advertising page sales used in current operating budgets at Time magazine. Actual quarterly predictions by Time executives for the years 1977-1981 are compared with predictions made by regression models that were based on data available to the executives when they made their predictions. Comparisons of executives and models are based on an accuracy criterion of actual absolute error. The present results agree with prior experimental results, which show that models predict more accurately than people. Therefore, one of the principal conclusions from lens model research-that simple models might successfully replace people in certain time-consuming, repetitive decision tasks-is supported. However, consistent underprediction by executives is observed. A crude correction for mean error is applied to predictions made by executives and by regression models, with the result that the executives’ corrected predictions are more accurate than the models’ corrected predictions.
Can a simple regression model outperform predictions of corporate executives when confronted with the task of estimating annual advertising page sales?
The author feeds the regression model the following variables:
- The quarter to which the data apply (first, second or third);
- Total advertising pages actually published in the magazine during the quarter
- Total advertising pages for the quarter in the alcoholic beverages
- Total advertising pages for the quarter in the automotive
- Total advertising pages for the quarter in the smoking materials categories
- New (i.e., incremental) bookings for the quarter
- Cumulative actual advertising pages for the year-to-date.
Table 1 highlights executives vs model results. * represent observations where the executives beat models. As you can see, there aren’t many “stars.” In general, executives underpredict (conservative bias?). In no cases are the prediction results by the executives better than the results from the models. Models win this round.
Thoughts on the paper?
Note: This site provides NO information on our value investing ETFs or our momentum investing ETFs. Please refer to this site.
Join thousands of other readers and subscribe to our blog.
Please remember that past performance is not an indicator of future results. Please read our full disclosures. The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Alpha Architect, its affiliates or its employees. This material has been provided to you solely for information and educational purposes and does not constitute an offer or solicitation of an offer or any advice or recommendation to purchase any securities or other financial instruments and may not be construed as such. The factual information set forth herein has been obtained or derived from sources believed by the author and Alpha Architect to be reliable but it is not necessarily all-inclusive and is not guaranteed as to its accuracy and is not to be regarded as a representation or warranty, express or implied, as to the information’s accuracy or completeness, nor should the attached information serve as the basis of any investment decision. No part of this material may be reproduced in any form, or referred to in any other publication, without express written permission from Alpha Architect.
Definitions of common statistics used in our analysis are available here (towards the bottom)