What makes reading comprehension test items difficult? An exploratory analysis

Abstract

This paper reports on a sub-study based on a bank of test items developed and calibrated for a computerized adaptive reading test for primary and secondary students in Singapore schools. In the main study (as reported in various papers by Lim Tock Keng et al, 1994, 1995), a bank of multiple-choice test items constructed for primary 3 and 5 and secondary 1 and 3 students was found to contain a relatively large number of test items that were quite difficult for these students. While generally it was feasible to guess the (rather obvious) reasons for the very easy items, it was thought necessary to check empirically on the factors that contribute to item difficulty. This sub-study attempts to answer directly the research question (What makes test items in reading comprehension difficult?) in relation to the items developed for the primary 5 students (beween 10 and 11 years old). This was done empirically from the perspectives of the linguistic features of the item texts, propositional analysis, decision processing, and the cognitive demand of the test items. The paper will be in five parts, namely, (a) a brief discussion of the cognitive model of text processing that underpins this exploratory analysis and the idea of text difficulty, (b) the operationalisation of the variables, (c) the statistical findings, (d) the sources of difficulty in English reading comprehension tests of the type studied, for students whose first language is not necessarily English, and (e) implications for test development.