Speaker

Dr. Klaas Sijtsma
Professor Emeritus, Department of Methodology and Statistics\ Tilburg University

Dr. Sijtsma is emeritus professor in the Department of Methodology and Statistics at Tilburg University, having previously served as Chair, Dean, and Rector Magnificus (equivalent to university president in US). He has held several national-level positions advising on educational and psychological testing (e.g., at CITO and COTAN), is a former president of the Psychometric Society (2010), and recipient of their lifetime achievement award (2023). He has published over 275 research papers and several books addressing many topics in psychometrics, although he is perhaps best known for his work on non-parametric item response theory. In particular, his work on the stochastic ordering properties of the sum score and critiques of CTT reliability coefficients have been widely cited.

Title

Pros and Cons of the Simple Sum Score on Tests and Questionnaires

Abstract

Since the introduction of latent variable models such as item response theory models and factor models (structural equation modeling) in psychometrics, latent variables have become favorite for representing abilities and traits and differentiating individuals with respect to these attributes. However, in the practice of reporting test results to examinees (and parents and teachers), job applicants (and employers), and patients (and therapists), sum scores and test scores that are transformations of sum scores (e.g., standardized scores, percentiles, IQ-scores, scores on educational tests) are often used because of their simplicity and intuitive appeal: They are simply easier to understand and interpret than scores on latent variables such as log odds, and therefore have greater potential for communication.

I will demonstrate that for standard testing, sum scores and their transformations are fully acceptable. Actually—and few people including psychometricians know this—item response theory (and not only the Rasch model) provides the theoretical justification for the practical use of the simple sum score. I will discuss situations in which the sum score should be preferred to the latent variable and illuminate the opposite situations where the latent variable is superior and must be preferred to the sum score. Both have merits and drawbacks, often in different situations. Thus, they complement and strengthen each other rather than compete, and there is no need to choose one and abandon the other.

Literature (freely downloadable):

• Sijtsma, K., Ellis, J. L., & Borsboom, D. (2024). Recognize the value of the sum score, psychometrics’ greatest accomplishment. Psychometrika, 89, 84-117. https://doi.org/10.1007/s11336-024-09964-7
• Sijtsma, K., Ellis, J. L., & Borsboom, D. (2024). Rejoinder to McNeish and Mislevy: What does psychological measurement require? Psychometrika, 89 (4), 1175–1185. https://doi.org/10.1007/s11336-024-10004-7