Language Testing And Validation
Validation in Language AssessmentAntony John Kunnan (1998)Mahwah, NJ: Lawrence Erlbaum AssociatesPp. xiii + 290ISBN 0-8058-2753-6 (paper)US $32.50 (also available in cloth, $59.95)Validation in Language Assessment is a collection of selected papers from the 17th Language Testing Research Colloquium. The volume opens with an introduction to approaches to validation in second and foreign language assessment in published research in the last 15 years. In this article, Antony John Kunnan asserts that the idea of validation has been the center of intense language assessment research. He sheds light on the foci of language assessment researchers by categorizing key studies according to the Messick framework (1989), which recommended that a unified validity framework be constructed. Messick's progressive matrix of validity details two interconnected facets of the unitary validity concept. One facet is the source of justification of testing, based on either appraisal of evidence or consequence. The other facet is the function of the test, being either interpretation or use. Kunnan's study reveals what he calls an "imbalance" in the attention researchers have given certain facets of this framework.
Language Testing and Validation
Kunnan's introductory chapter is followed by 11 chapters that are presented in 3 parts: Part I presents four papers that focus on validation through the stages of the test development and test-taking process. Part II presents six papers that focus on validation by examining data from test-taker characteristics and test-taker feedback. Part III presents an analytical assessment of the presentations at past Language Testing Research Colloquiums.
Dorry Kenyon's chapter is the lead article in Part I, which illustrates the conventional approach to assessment validation research. Kenyon investigates foreign language students' perceived difficulty in performing various speaking tasks in the American Council on the Teaching of Foreign Languages (ACTFL) Speaking Proficiency Guidelines hierarchy. The purpose of her study is to examine and shed light on the validity of the demands of oral proficiency tasks constructed according to the criteria contained in the ACTFL Guidelines.
Three chapters on test development follow, each one focusing on a single assessment concern: Read on a new test format for vocabulary; Fortus, Coriat, & Fund on item difficulty in reading comprehension; and Wiggelsworth on time planning in language assessment.
John Read's study is a contribution to the ongoing validation of the word associates format, a selected-response type of test item designed to measure depth of vocabulary knowledge. Read draws on test performance data from New Zealand to provide evidence for concurrent validity of the new format. [-1-]
Gillian Wigglesworth's study focuses on an important aspect of the test-taking process: the presence or absence of planning time. Her paper describes a study of the effects of planning time on second language oral test discourse in an oral interaction test in Australia. She uses techniques from discourse analysis to examine the nature and significance of differences in the areas of fluency, accuracy,and complexity in the second language.
Section II of Validation in Language Assessment presents six papers that focus on validation by examining test-taker characteristics and feedback. James Purpura's paper details the development and construct validation of a questionnaire that measures the reported cognitive strategy and cognitive background characteristics of test takers in the United States. Purpura's primary goal is to design an instrument that allows test takers to report the cognitive strategies they think they use in second language acquisition, use, and testing so that these processes can ultimately be related to second language test performance.
Caroline Clapham's chapter examines the effect of language proficiency and background knowledge on students' reading comprehension in the United Kingdom. The aim of Clapham's study is to investigate the effects of background knowledge on reading comprehension, and to examine whether students should be given reading proficiency tests in their own academic content areas.
April Ginther and Joseph Stevens investigate the internal construct validity of an advanced Spanish-language placement exam in order to determine whether the traditional four-factor examination structure (listening, speaking, reading, and writing) was invariant for certain subpopulations. Ginther and Stevens analyze data drawn from Latinamerican Spanish-speaking examinees as well as Mexican Spanish-speaking, Mexican Spanish-English bilinguals, White English-speaking and Black English-speaking examinees.
Annie Brown and Noriko Iwashita's paper examines the role of native language background in the validation of a computer-adaptive test. Their study investigates the performance of learners of Japanese from different language backgrounds (native speakers of English, Chinese, and Korean) on a 225-item multiple-choice question computer-adaptive grammar test.
Kathryn Hill investigates the effect of test-taker characteristics on reactions to an oral English proficiency test called the Access Test, used to assess the English language proficiency for prospective migrants to Australia. Hill seeks reactions to the test's validity by means of a questionnaire.
Bonny Norton and Pippa Stein's chapter addresses issues of textual meaning, testing, and pedagogy on the basis of data drawn from piloting a college reading exam in English for black students in South Africa. Their findings call into question a number of assumptions about language and language assessment. [-2-]
Liz Hamp-Lyons and Brian Lynch provide what Kunnan calls "a fitting conclusion" to a volume on validation in language assessment. They examine research practices of the second- and foreign-language testing community as seen through the abstracts of papers presented at the Language Testing Research Colloquium throughout the last 15 years. Hamp-Lyons and Lynch focus their analysis on the ways in which test validity and reliability have been addressed in language testing research. This interesting concluding chapter explores the extent to which the Language Testing Research Colloquium community has already engaged itself with newer modes of inquiry beyond the psychometric.
Validation in Language Assessment is an essential read for those working in the language testing community. Kunnan has done an admirable job in selecting the studies represented in this volume as they present diverse approaches to test validity from an international perspective. A particularly valuable part of the book is the annotated list of suggested readings presented at the end of each chapter. Kunnan's volume is a valuable addition to the body of knowledge in language assessment.
With over 30 years in the language services business, ALTA has built a reputation as a trusted provider of valid and reliable language tests. Our clients depend on us to help them create defensible assessment programs whether through the use of our standard language tests, or through the customization of tests specifically for their companies, organizations, or agencies.
It is important to note that many clients will use the standard tests, which have already undergone this process of validation, and simply choose to employ ALTA to set their cut-score through the Job Analysis and administer the test. This is because the definition of a minimally qualified candidate can change from job to job, or from one organization to another.
The series addresses a wide range of important issues and new developments in language testing and assessment, and is an indispensable resource for test users, developers and researchers. There are currently over 50 titles available; a full list of these, plus content summaries, is provided below. For a reader's overview of the series, including a thematic categorisation and extracts from reviews, please see this essay, kindly contributed to Cambridge English by a visiting professor, Xiangdong Gu:Download Studies in Language Testing Essay by Xiangdong Gu (PDF)
The volume is a rich source of information on all aspects of testing academic reading ability. Examination boards and other institutions who need to validate their own academic reading tests in a systematic and coherent manner, or who wish to develop new instruments for measuring academic reading, will find it a useful resource, as will researchers and graduate students in the field of language assessment, and teachers preparing students for IELTS (and similar tests) or involved in English for Academic Purposes (EAP) programmes.
Written by a selection of his friends and collaborators, this volume pays tribute to the academic achievements of the late Professor Cyril J Weir. His passing in September 2018 leaves an eclectic legacy in the field of language testing and assessment, and the chapters contained herein, part of a series he guided and often wrote for, honour and illuminate his lessons.
This volume takes a framework for validating tests that was developed in language testing, and applies it to an admissions test used for biomedical courses. The framework is used to consider validity in the BioMedical Admissions Test (BMAT). Each chapter focuses on a different aspect of validity and also presents research that has been conducted with the test. By addressing all of the validity aspects identified as important by language testers, this volume presents a comprehensive evaluation of BMAT's validity. The processes of evaluation used in the book also promote a cross-disciplinary approach to assessment research, by demonstrating how effectively language testing frameworks can be used in different educational contexts. The authors of the chapters include Cambridge Assessment staff and medical education experts, from a wide range of subject backgrounds. Psychologists, clinicians, linguists and assessment experts have all contributed to the volume, making it an example of multidisciplinary collaboration. 041b061a72