Aryadoust, Vahid

Structural equation modeling (SEM) has been shown to be an effective and powerful tool in language testing and assessment as it offers advantages over fundamental statistical techniques such as regression and correlation analysis. Specifically, SEM is capable of confirming multiple hypothesized direct and indirect relationships among different variables in one analysis. For example, it can be used to identify the relationship of task characteristics with test takers' performance or to explore the relationship of a test and test takers. It can also be used to confirm the factor structure of tests. This chapter provides an overview of the key concepts in SEM analysis, identifies five stages in SEM analysis, and provides an overview of language assessment studies that have utilized this method. An application of this method is shown through a sample study that aims to determine whether test scores on an English diagnostic test of receptive skills can predict performance in an English proficiency test of receptive and productive skills and whether academic background impacts these test scores. The chapter concludes with a discussion on the benefits and limitations of SEM in language testing and assessment research.

The Academic Listening Self-rating Questionnaire (ALSA) is a 47-item self-appraisal tool that helps language learners evaluate their own academic listening skills (Aryadoust, Goh, & Lee, 2012). The six underlying dimensions of the ALSA consist of (a) linguistic components and prosody, (b) cognitive processing skills, (c) relating input to other materials, (d) notetaking, (e) memory and concentration, and (f) lecture structure. The psychometric quality of ALSA has been studied using the Rating Scale Rasch model, structural equation modeling, and correlation analyses. The ALSA can be used to raise tertiary-level students’ awareness of their academic listening ability and of the elements of academic discourse, such as lectures and seminars, that may affect their academic achievement. Further research is being undertaken to provide validity evidence for two versions of the instrument in Chinese and Turkish, respectively.

This chapter focuses on the log-linear cognitive diagnosis modeling (LCDM), a general diagnostic classification model (DCM) family that allows researchers to model a large group of diagnostic classification models (DCMs) flexibly. Although the LCDM has important advantages over other core DCMs, it remains relatively under-researched in language assessment. This chapter first provides language testers with an introduction to the theoretical and statistical underpinnings of the LCDM. Next, it demonstrates how the LCDM could be applied to a high-stakes listening comprehension test. Finally, it presents guidelines on how to estimate and interpret the model, item, and examinee parameters with readily available software.

This chapter introduces evolutionary algorithm-based (EA-based) symbolic regression, which is an optimization model inspired by nature. EA-based symbolic regression is used to predict reading comprehension proficiency by using English learners' vocabulary and grammatical knowledge. EA-based symbolic regression draws on the fundamental concepts of Darwinian evolution, such as breeding and variety, and applies modeling to assess the accuracy and relevance of the prediction models. In this technique, multiple models are generated among which the one with the optimal fit is chosen as the “parent” and the basis for “breeding” further models, called offspring, for the following generations. The present study finds a significant nonlinear relationship between lexicogrammatical knowledge and reading comprehension proficiency (R2 = .520). Details and computational requirements are discussed and implications for language assessment are explored.

This chapter introduces classification and regression trees (CART) to investigate the factors that determine listening test item difficulty. CART has been used extensively in education fields and machine learning but has received relatively less attention in language assessment. We discuss the major components of a reliable CART study such as articulation of theoretical frameworks, using cross-validation, estimation of fit statistics, and accuracy of classification. The chapter employs the construction-integration (CI) model of comprehension to measure item difficulty in a large pool of listening test items. The data comprised seven Michigan English Test (MET) listening tests comprising 321 items answered by 5039 international language learners. CART modeling generated 41 IF-THEN rules, which revealed nonlinear relationships between item difficulty and 12 independent variables (IVs) measured by Coh-Metrix. CART enabled us to show that the relationship between item difficulty and the 12 IVs is not linear, and different sets of rules would apply to predict the difficulty of different groups of listening test items.

This chapter examines approaches to second language acquisition (SLA) in relation to L2 listening development and assessment. As researchers increasingly recognize the limitations of a purely cognitive-behavioral theory of SLA, socially-oriented approaches to SLA have gained momentum in the field. Thus, the scope of this chapter will extend beyond the cognitive and behavioral approaches and will incorporate sociocultural theory and complexity theory in SLA in the context of L2 listening development. It aims to present evidence supporting the role of exposure to spoken input in language development and discusses the role of extensive listening in L2 learning.

Strategies-based instruction (SBI) is widely accepted and successfully implemented in North America in language and literacy programmes, but little has been reported on how this strategy would work in a bilingual/biliteracy learning context. This chapter reports on the efficacy of such an intervention conducted in two Singapore primary schools, where the government implements a unique bilingual/biliteracy policy in education, by which English is offered as the first language and one of the other three mother tongue languages (Chinese, Malay and Tamil) as a second language subject in the national curriculum. Although the Singapore quadrilingual education policy has been internationally acclaimed as being successful, some students face challenges in biliteracy learning, resulting in some students’ underachievement. To help these students catch up with their better-performing peers, we designed an intervention programme to answer the following research questions: (1) When integrated into the regular curriculum, does SBI have an impact on bilingual students’ understanding of the writing processes in their two languages? (2) Specifically, does SBI lead to writing improvement in both languages? The study had an experimental group and a control group. Such a design was intended for comparing the pedagogical efficacy of SBI on student improvement in writing in English and writing in Chinese over a period of one semester (10 weeks of teaching) in the regular school curriculum. Results suggest that the use of SBI not only raised students’ awareness of writing strategies but also improved their English and Chinese writing scores. Thus, we conclude that SBI was a useful dimension to the writing curriculum in the two schools involved in this study.