Viewing behavior and language background in listening assessment : an eye-tracking and mouse-tracking study

Listening is a multidimensional construct involving cognitive, behavioral, and physiological processes. Given the intricacy and complexity of listening comprehension, previous studies have identified several test-specific and listener-specific factors that significantly impact and predict listening test performance. The main goal of this study is to explore two of these factors in a large-scale listening test, the International English Language Testing System (IELTS). It has been indicated by test developers that the difficulty of the IELTS listening section increases progressively; that it is unbiased for all candidates from various language backgrounds; and that the test measures listening skills. To validate these assertions, this study employed two sensor technologies, eye-tracking and mouse-tracking, to explore the impact of test-specific and listener-specific factors on listening performance. Specifically, it examined the differences in viewing behaviors and mouse behaviors across the four listening sections and between English-as-first language (E-L1) and English-as-second language (E-L2) listeners. Furthermore, the predictive power of gaze behaviors, mouse clicking, and language background on listening test scores was investigated.

The study involved 77 participants, 45 E-L1 candidates and 32 E-L2 candidates, from a major university in Singapore. Participants’ gaze and mouse behaviors were recorded when they completed a computerized listening performance test. The collected data were analyzed using repeated measures MANOVA and linear mixed models (LMMs). Three key findings were: 1) IELTS listening test was progressively difficult across the four sections and E-L1 candidates scored significantly higher than E-L2 candidates in all four sections and this disparity became increasingly pronounced; 2) significant differences were found in gaze behaviors and mouse clicking across the four sections and between E-L1 and E-L2 listeners on both test items and instructions; 3) gaze behaviors, mouse behaviors, and language background were significant predictors of listening test performance, explaining 33.2% of the variance observed in the candidates’ test scores.

This study offers both theoretical and empirical implications for listening assessment. It contributes to the understanding of the listening construct in large-scale tests and evidences the impact of test-specific and listener-specific factors. Assumptions of test developers and implications for listening assessment and pedagogy are discussed.