Assessing secondary school students' growth in mathematics problem solving : an item response theory approach

Abstract

The use of assessment to measure growth of academic achievement is a key feature in educational institutions. Critical decisions are made from interpretations of these assessments which has far reaching consequences for students, educators and school administrators alike. This has necessitated the utilisation of psychometric evidence to provide accurate and fair ways to interpret scores from these assessments in describing growth of students’ achievement between critical points of their education. Traditionally, Classical Test Theory (CTT) test equating methods were used to ensure the equity of scores from different forms of the same test taken by different group of examinees. However, CTT methods are item and sample dependent which is a major limitation in the area of assessment where invariant items and person statistics are needed. Advancements in Item Response Theory (IRT) has provided educators and school administrators alternative methods in the area of educational assessment. In this study, the IRT test equating method was examined for its appropriateness in measuring the growth of Mathematics problem solving ability of 79 secondary school students as they advance from Secondary One to Two. The Rasch Partial Credit Model (PCM) was used to determine data-model fit, Differential Item Functioning (DIF) and the fulfilment of IRT assumptions. Results show that all assumptions of IRT were satisfied and the data-model fit conditions were met. The Mantel-Haenszel (MH) and Rasch-Welch t-test results also showed that all items displayed negligible DIF. The abovementioned results indicated that the Rasch PCM is an appropriate model to be used in assessing the growth of Mathematics problem solving ability.