Psychometric properties of a reading comprehension test for English-Chinese bilingual children in Singapore: A pilot study

Reading comprehension (RC) is essential for academic success and everyday functioning, making its accurate assessment crucial for identifying students needing support and guiding educational interventions. However, existing standardised RC assessments, primarily developed in monolingual English contexts, have limitations when used with students with diverse language experiences, such as those in Singapore. This study examined the psychometric properties of the English and Chinese versions of the Singapore Multilingual Assessment of Reading Comprehension (SMARC), a RC test designed for Singaporean bilingual children by O’Brien et al. (2021).

Secondary data collected from 436 Singapore primary school students across grades Primary 1 to 3 (P1 to P3), who each completed the RC test corresponding to their grade level in English and their mother tongue language, were analysed using the dichotomous Rasch model. Test validity was evaluated based on (i) unidimensionality (i.e., whether items measure the same latent construct), (ii) the fit between observed data and the Rasch model (i.e., whether higher-achieving test takers have higher probability of answering individual items correctly than lower-achieving test takers), and (iii) the targeting of items (i.e., how well the test difficulty matched person ability). Test reliability was evaluated based on the test’s discriminative ability (i.e., how well test takers were classified into different levels of RC ability).

Findings revealed that the tests currently do not meet the requirements to serve as a reliable and valid measurement tool at their intended grade levels. Analyses presented in this paper exhibited low validity and reliability for the English and Chinese P1 and P2 tests, and low validity with unacceptable reliability for the English and Chinese P3 tests. While the English and Chinese P1 and P2 tests demonstrated potential for use as screeners - identifying below-average performers in the English tests, and broadly distinguishing higher- and lower-performing groups in the Chinese tests—their accuracy would remain questionable. Recommendations to refine the test were provided based on observations that the English tests were too easy for their respective grade levels while the Chinese P3 test was somewhat easy. Additionally, the Chinese P1 and P2 tests had items that impacted the overall fit of the tests.

Overall, this pilot study provided preliminary insights into the psychometric properties of the first RC test developed for bilingual Singaporean children, highlighting areas for improvement. Further research is essential to refine the RC test, ultimately creating an operational tool that can facilitate bilingual research and diagnostic applications, thereby to enhance reading development and educational outcomes in Singapore.