Please use this identifier to cite or link to this item: http://hdl.handle.net/10497/18553
Title: 
Authors: 
Keywords: 
Assessment
Oral proficiency
Rater judgement
Rater perception
Speaking
Testing
Issue Date: 
2011
Citation: 
Ang-Aw, H. T., & Goh, C. C. M. (2011). Understanding discrepancies in rater judgement on national-level oral examination tasks. RELC Journal, 42(1), 31-51. http://dx.doi.org/10.1177/0033688210390226
Abstract: 
The oral examination is an important component of the high-stakes ‘O’ level examination in Singapore taken by 16-17 years olds whose first language may or may not be English. In spite of this, there has been sparse research into the examination. This paper reports findings of an exploratory study which attempted to determine whether there were any discrepancies in rater judgements and thereafter, explore the nature and scope of the discrepancies identified. Five audio recordings were obtained from a simulated oral examination of five candidates conducted by a trained ‘O’ level oral examiner. Seven other trained ‘O’ level oral examiners were asked to rate four of the recordings individually and provide concurrent verbal reports. Questionnaires were also given to the raters for data triangulation after the verbalisation. The data were analysed through Verbal Protocol Analysis and descriptive statistics. Rater discrepancies detected in the scores were qualitatively determined to be due to four differences: emphases on factors assessed, constructs of oral proficiency, rater interpretations and approaches in assessment. These findings provide valuable insights into raters' perceptions of the construct of speaking and offer implications for rater training and the development of rating scales.
Description: 
This is the final draft, after peer-review, of a manuscript published in RELC Journal. The published version is available online at http://journals.sagepub.com/doi/pdf/10.1177/0033688210390226
URI: 
ISSN: 
1090-4018
Other Identifiers: 
10.1177/0033688210390226
Appears in Collections:Journal Articles

Files in This Item:
File Description SizeFormat 
RELCJ-42-1-31.pdf511.97 kBAdobe PDFThumbnail
View/Open
Show full item record

Page view(s) 50

52
Last Week
0
Last month
2
checked on May 26, 2019

Download(s) 50

97
checked on May 26, 2019

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.