Now showing 1 - 10 of 58
  • Publication
    Open Access
    A Rasch analysis of an international English language testing system listening sample test
    This study reports on an investigation of the construct validity of an International English Language Testing System (IELTS) Listening sample test. The test was administered to 148 multinational participants. The Rasch modeling of data was used to fulfill the research objectives. Four major conclusions were made: 1) the Rasch differential item functioning analysis revealed that limited production items behave differently across different test taker groups suggesting the presence of construct-irrelevancies, 2) multiple choice questions (MCQ) do not cause construct-irrelevancies unless testees need to make ‘close paraphrases’ to comprehend the item stem or the question demands more than one answer; this nominates short MCQ as a best item format in listening tests, 3) evidence was found for ‘lexical processing’ which is different from top-down/bottom-up processing, and 4) the Wright map provided evidence for construct under-representation of the test. Findings from this study provide different sorts of evidence supporting and disproving the claim of the construct validity of the test, although they should be further investigated in future studies with different samples. Implications of the findings for IELTS and item writers are also discussed.
      223  489
  • Publication
    Metadata only
    Investigating the construct validity of the MELAB Listening Test through the Rasch analysis and correlated uniqueness modeling
    (University of Michigan, 2010) ;
    This article evaluates the construct validity of the Michigan English Language Assessment Battery (MELAB) listening test by investigating the underpinning structure of the test (or construct map), possible construct under representation and construct-irrelevant threats. Data for the study, from the administration of a form of the MELAB listening test to 916 international test takers, were provided by the English Language Institute of the University of Michigan. The researchers sought evidence of construct validity primarily through correlated uniqueness models (CUM) and the Rasch model. A five factor CUM was fitted into the data but did not display acceptable measurement properties. The researchers then evaluated a three-traits1 confirmatory factor analysis (CFA) that fitted the data sufficiently. This fitting model was further evaluated with parcel items, which supported the proposed CFA model. Accordingly, the underlying structure of the test was mapped out as three factors: ability to understand minimal context stimuli, short interactions, and long-stretch discourse. The researchers propose this model as the tentative construct map of this form of the test. To investigate construct under representation and construct-irrelevant threats, the Rasch model was used. This analysis showed that the test was relatively easy for the sample and the listening ability of several higher ability test takers were sufficiently tested by the items. This is interpreted to be a sign of test ceiling effects and minor construct-underrepresentation, although the researchers argue that the test is intended to distinguish among the students who have a minimum listening ability to enter a program from those who don’t. The Rasch model provided support of the absence of construct-irrelevant threats by showing the adherence of data to uni dimensionality and local independence, and good measurement properties of items. The final assessment of the observed results showed that the generated evidence supported the construct validity of the test.
      32  261
  • Publication
    Open Access
    A corpus study of language simplification and grammar in graded readers
    (2023)
    Azrifah Zakaria
    ;
    ;
    Studies on graded readers used in extensive reading have tended to focus on vocabulary. This study set out to investigate the linguistic profile of graded readers, taking into account both grammar and lexis. A corpus of 90 readers were tagged according to the variables in Biber’s Multidimensional (MD) analysis, using the Multidimensional Analysis Tagger (MAT). These variables were analysed using latent class cluster analysis to determine whether the graded readers can be grouped by similarity in linguistic features. While MAT analysis surfaced more similarities than differences within the corpus, latent class clustering produced an optimal 3-class model. Post-hoc concordance analyses showed that graded readers may be categorised as having three classes of complexity: beginner, transitional, and advanced. The findings in the study suggest that selection of reading materials for extensive reading should take into consideration grammatical complexity as well as lexis. The linguistic profiles compiled in this study detail the grammatical structures and the associated lexical items within the structures that teachers may expect their students to encounter when reading graded readers. In addition, the profiles may be of benefit to teachers seeking to supplement extensive reading with form-focused instruction.
      33  43
  • Publication
    Restricted
    Cognitive Diagnostic Assessment System (CoDiAS) for Singapore’s secondary schools: Toward individualized learning and assessment in language education
    (Office of Education Research, National Institute of Education, Singapore, 2023) ;
    To date, several computerized diagnostic systems have been developed. These systems are limited in their feedback delivery and assessment scopes as well as in the delivery of remedial programs. For example, the Diagnostic English Language Needs Assessment (DELNA) developed by the University of Auckland and the Diagnostic English Language Assessment (DELA) designed by the University of Melbourne function primarily like placement tests where feedback is delivered to the learners but the tests aim to place students in different language learning programs (of course, one could argue that this is the treatment that follows the diagnosis, albeit not highly differentiated at the individual level.). Similarly, the Diagnostic English Language Tracking Assessment (DELTA) designed by Hong Kong Polytechnic University and the Diagnostic Language Assessment (DIALANG) produced by Lancaster University provide feedback to learners but without specifying skill mastery profiles, differentiated remedial programs, or actionable plans (Harding, Alderson, & Brunfaut, 2015). These systems are also limited by their inability to provide fine-grained information on learners’ growth over time.
      12  51
  • Publication
    Open Access
    An introduction to the cognitive diagnostic assessment system
    (National Institute of Education (Singapore), 2020) ;
      66  66
  • Publication
    Open Access
      135  136
  • Publication
    Metadata only
    Developing and validating an academic listening questionnaire
    (Pabst Science, 2012) ; ;
    Lee, Ong Kim
    This article reports on the development and administration of the Academic Listening Self-rating Questionnaire (ALSA). The ALSA was developed on the basis of a proposed model of academic listening comprising six related components. The researchers operationalized the model, subjected items to iterative rounds of content analysis, and administered the finalized questionnaire to international ESL (English as a second language) students in Malaysian and Australian universities. Structural equation modeling and rating scale modeling of data provided content-related, substantive, and structural validity evidence for the instrument. The researchers explain the utility of the questionnaire for educational and assessment purposes.
      6
  • Publication
    Open Access
    The known and unknown about the nature and assessment of L2 listening
    Although second language (L2) listening assessment has been the subject of much research interest in the past few decades, there remain a multitude of challenges facing the definition and operationalization of the L2 listening construct(s). Notably, the majority of L2 listening assessment studies are based upon the (implicit) assumption that listening is reducible to cognition and metacognition. This approach ignores emotional, neurophysiological, and sociocultural mechanisms underlying L2 listening. In this paper, the role of these mechanisms in L2 listening assessment is discussed and four gaps in understanding are explored: the nature of L2 listening, the interaction between listeners and the stimuli, the role of visuals, and authenticity in L2 listening assessments. Finally, a review of the papers published in the special issue is presented and recommendations for further research on L2 listening assessments are provided.
    Scopus© Citations 5  91  29
  • Publication
    Open Access
    A functional analysis of the dialogues in the new interchange intro textbook
    This study investigates language functions in the New Interchange Intro textbook. It is observed that more diffi cult grammar structures, e.g., wh-questions, have a gradual increase throughout the dialogues, but simple yes/no questions and statements are more pronounced in the opening lessons. Also, declarative sentences outnumber other grammar functions, and wh-questions rank second. Grammar structures establish three major macro-pragmatic functions: representative, directive, and expressive. However, commissive and declaration functions are not observed in the dialogues, which can be due to their diffi culty for new learners to communicate these language functions. Among micros, stating greetings, requesting, and expressing gratitude have the least frequency whereas exchanging information has the greatest frequency. Naturally, research on language functions in English textbooks helps teachers in selecting the best materials.
      172  197