1 Introduction 1
Definitions: Applied linguistics, evaluation, program 1
Critical issues 2
Program evaluation and applied linguistics research 9
2 Historical background 12
The paradigm dialog 12
Evaluation of language education programs in the 1960s and 1970s 22
More recent developments in the evaluation of language education programs 32
Summary 39
3 Validity 41
Validity from the positivistic perspective 41
Validity from the naturalistic perspective 53
Conclusions: Validity from the two perspectives 66
4 Positivistic designs 70
True experimental designs 71
Quasi-experimental designs 73
5 Naturalistic designs 80
The responsive model 80
The illumination model 82
Goal-free evaluation 84
The judicial model 85
The connoisseurship model 86
Other metaphors for naturalistic evaluation 88
Data gathering 92
6 Quantitative data gathering and analysis 92
Data analysis 94
Conclusion 105
7 Qualitative data gathering and analysis 107
Overview 107
Data gathering 108
Data analysis 139
8 Combining positivistic and naturalistic program evaluation 155
Compatibilist versus incompatibilist perspectives 155
Mixed strategies 156
Multiple strategies 158
Mixed designs 159
Mixed designs and strategies over time 165
9 Conclusions 167
CAM step 1 (audience and goals): Determine the purpose of the evaluation 167
CAM steps 2 and 3 (context inventory and preliminary thematic framework): Determine what is being evaluated 170
CAM steps 4 and 5 (evaluation design and data collection):Select a design and collect the data 171
CAM step 6 (data analysis): Analyze and interpret your findings 173
CAM step 7 (evaluation report): Communicating the evaluation findings 174
The role of program evaluation in applied linguistics research 175
References 178
Author index 188
Subject index 191