An eye-tracking investigation of the keyword-matching strategy in listening assessment

Abstract

Studies have shown that test-takers tend to use keyword-matching strategies when taking listening tests. Keyword-matching involves matching content words in the written modality (test items) against those heard in the audio text. However, no research has investigated the effect of such keywords in listening tests, or the impact of gazing upon these keywords on listening test scores. Thus, this study examined whether test-takers’ performance on a listening test can be explained by their gaze behaviors across three types of content words in the written modality: nouns, verbs, and adjectives. Using eye-tracking technology, this study measured the gaze behavior of 66 listening test-takers during reading content words in test item stems. Using linear mixed effect model, binary probit regression, and multinomial logistic regression, we found that test-takers’ performance was predicted by gaze behavioral measures on content words. Among the content words, fixating on nouns in written test items had the most significant role in predicting test performance, followed by adjectives and verbs. By shedding light on how keywords in test items are attended to by test-takers and the relationship between keyword-matching and listening test performance, this study has provided significant evidence for the overwhelming role of reading in listening tests. Implications for test score interpretation are discussed.