Issues and Methods in Evaluating Differences between Models

Conference on Modeling Developmental Processes in Ecological Context

Tempe, Arizona

March 2004

Abstract of talk by Robert MacCallum, University of North Carolina Chapel Hill:

In applications of structural equation modeling and related methods it is common for researchers to seek to evaluate competing alternative models and to identify which among two or more models is optimal in some sense. For comparing nested models, the standard procedure is to employ the likelihood ratio test of the difference in model fit, where the null hypothesis is that the two models fit equally well in the population. A procedure for determining the statistical power of this test will be presented, and factors that affect statistical power will be delineated. In addition, a modification of the standard null hypothesis of zero difference in fit will be proposed. It can be argued that in practice this null hypothesis is virtually never true and is empirically uninteresting. A modified testing procedure will be presented that allows for a null hypothesis of a specified small difference in fit, versus an alternative hypothesis of a larger difference, along with corresponding power analysis procedures. Finally, some additional issues regarding model comparison will be discussed briefly, including a proposed procedure for using residual matrices to compare non-nested models, as well as the role of model complexity in the model comparison and selection problem.