Options
A multifarious approach to test equity in listening assessments : use of linguistic features, test scores, eye-tracking and neuroimaging data
Author
Lucio, Ester Dominguez
Supervisor
Aryadoust, Vahid
Abstract
Listening comprehension is a complex process that involves different steps. Listeners need to extract meaning from an oral and/or visual stimuli from the speaker. Listeners also need to decode that message into smaller pieces and re-code using their own previous knowledge, to help them to create a mental representation. Traditionally these listening processes have been assessed using listening tests. As tests often have different forms, they should be based on the same content and specifications, so that they can have interchangeable scores. In order to achieve the parity in test forms, test developers use a technique called equating tests.
For equating tests, developers traditionally take into consideration only test scores of the participants and the following features: error, properties, and sample size, without taking into consideration the neurobiological aspects of the test-takers. This can present a problem because test-takers can use different strategies to pass the test, so the scores are just a mere representation of these strategies, not the cognitive processes involved in listening. To address this gap, in the present study, equating was presented at four levels: (i) linguistic features of the test contents, (ii) test scores, (iii) gaze behaviours and (iv) neurocognitive behaviours. It was found that linguistic features were similar in both tests and there was no significant difference in test scores and neurocognitive data. However, there was a significant difference in gaze behaviour (fixation count and visit count).
The results of this dissertation can help test developers to equate tests not only based on the test-takers' scores but in the content of the text and the neurobiological processes that test-takers are engaged in. This would help to satisfy one of the important requirements of validation underscored by Messick (1979) which is the investigation of validity across multiple test forms.
For equating tests, developers traditionally take into consideration only test scores of the participants and the following features: error, properties, and sample size, without taking into consideration the neurobiological aspects of the test-takers. This can present a problem because test-takers can use different strategies to pass the test, so the scores are just a mere representation of these strategies, not the cognitive processes involved in listening. To address this gap, in the present study, equating was presented at four levels: (i) linguistic features of the test contents, (ii) test scores, (iii) gaze behaviours and (iv) neurocognitive behaviours. It was found that linguistic features were similar in both tests and there was no significant difference in test scores and neurocognitive data. However, there was a significant difference in gaze behaviour (fixation count and visit count).
The results of this dissertation can help test developers to equate tests not only based on the test-takers' scores but in the content of the text and the neurobiological processes that test-takers are engaged in. This would help to satisfy one of the important requirements of validation underscored by Messick (1979) which is the investigation of validity across multiple test forms.
Date Issued
2021
Call Number
P95.46 Luc
Date Submitted
2021