Chapter 1 Introduction 1
1.1 Need for the study 3
1.1.1 Students'perspective 3
1.1.2 Teachers'perspective 4
1.1.3 Purpose of the study 5
1.2 Configuration of the study 6
1.2.1 Study procedure 6
1.2.2 Study questions 6
1.3 Significance of the study 8
1.4 Layout of the book 9
1.5 Summary 11
Chapter 2 Literature Review 12
2.1 Fundamentals of oral proficiency 13
2.1.1 Speech communication and its characteristics 13
2.1.2 Essence of speech communication 15
2.1.2.1 Language use versus usage 15
2.1.2.2 Communicative competence versus linguistic competence 17
2.1.2.3 Accuracy versus fluency 18
2.1.2.4 Correctness versus appropriateness 20
2.1.2.5 Proficiency versus achievement 21
2.1.2.6 Process versus product 21
2.1.2.7 Synthetic approach versus analytic approach 22
2.1.2.8 The learner versus the teacher 22
2.1.3 Components of oral proficiency 23
2.2 Classification of oral rating scales 26
2.2.1 Holistic rating scales 26
2.2.2 Analytic rating scales 29
2.2.3 Combination of holistic ＆ analytic rating scales 32
2.2.4 Integration of holistic ＆ analytic rating scales 40
2.2.5 Using analytic rating scales for the project 41
2.3 Summary 42
Chapter 3 Study Bases 44
3.1 Empirical basis 44
3.1.1 Syllabus description 44
3.1.2 Oral tests of CET ＆ TEM 48
3.2 Theoretical basis 51
3.2.1 CLA model 51
3.2.1.1 Language competence 51
3.2.1.2 Strategic competence 54
3.2.1.3 Psychophysiological mechanisms 55
3.2.1.4 Significance of CLA to OARS 55
3.2.1.5 Trait factors of CLA 57
3.2.2 Cohen's rating scale for pragmatic speaking 58
3.2.3 Nunn's rating scales for small group interaction 59
3.2.3.1 Nunn's rating scales of interactive skills 60
3.2.3.2 Nunn's rating scales of intelligibility 62
3.2.3.3 Significance of Nunn's rating scales to OARS 64
3.2.4 Reconsiderations on strategic competence 65
3.2.4.1 Classification of strategic competence 65
3.2.4.2 Evaluation of nonverbal communication 66
3.3 Summary 68
Chapter 4 Construction of OARS 69
4.1 Needs analysis 69
4.1.1 Basic considerations on band descriptors 69
4.1.2 Questionnaire construction 71
4.1.2.1 The trial questionnaire 71
4.1.2.2 Demographic analysis 75
4.1.2.3 Data analysis 76
4.1.3 Survey results 79
4.1.3.1 Demographic analysis 79
4.1.3.2 The OARS questionnaire 84
4.1.3.3 Data analysis 87
4.2 Description of OARS 95
4.2.1 Framework of OARS 95
4.2.2 Band descriptors of OARS 98
4.2.2.1 Band descriptors of organizational competence 99
4.2.2.2 Band descriptors of pragmatic competence 102
4.2.2.3 Band descriptors of flexible interaction 104
4.2.2.4 Band descriptors of nonverbal communication 105
4.3 Sample rating 107
4.3.1 Sample rating Ⅰ—role-play 108
4.3.1.1 Performance analysis 108
4.3.1.2 Suggested rating 110
4.3.2 Sample rating Ⅱ—discussion 114
4.3.2.1 Performance analysis 114
4.3.2.2 Suggested rating 116
4.4 Summary 121
Chapter 5 The Pilot Validation Study 123
5.1 Research purpose 123
5.2 Research subjects and task 124
5.3 Research operation 125
5.4 Data analysis 127
5.4.1 Prior consideration 127
5.4.1.1 Candidates'perspective 127
5.4.1.2 Teachers'perspective 129
5.4.2 The profound study 133
5.4.2.1 Analysis of reliability estimates 134
5.4.2.2 Analysis of descriptive statistics 136
5.4.2.3 Analysis of construct validity 137
5.4.2.4 Analysis of external validity 139
5.5 Discussion 142
5.6 Summary 145
Chapter 6 The Field Validation Study 146
6.1 Research purpose 146
6.2 Research subjects and task 147
6.3 Research operation 149
6.3.1 Process of experimental tests 149
6.3.2 Rater training 150
6.3.3 Collection of data 151
6.4 Data analysis 152
6.4.1 Study on six types of candidates 152
6.4.1.1 Analysis of reliability estimates 152
6.4.1.2 Analysis of descriptive statistics 153
6.4.1.3 Analysis of construct validity 154
6.4.1.4 Analysis of external validity 154
6.4.2 Study on 180 candidates 158
6.4.2.1 Analyses of reliability estimates 160
6.4.2.2 Analyses of descriptive statistics 160
6.4.2.3 Analyses of construct validity 161
6.4.2.4 Analyses of external validity 161
6.4.3 Characteristics of oral English proficiency of Chinese college students 164
6.4.4 Comparability study on different types of candidates 167
6.5 Discussion 172
6.6 Summary 178
Chapter 7 Conclusions and Recommendations 180
7.1 Validation results 180
7.2 Washback effects 182
7.2.1 Design of OARS as a model 183
7.2.2 Precautions in application of OARS 187
7.2.3 Contribution of OARS to oral instruction 189
7.3 Existing problems 190
7.3.1 Problems with OARS itself 190
7.3.2 Problems with the validation study 191
7.4 Further research 192
7.5 Summary 193
Bibliography 194
Appendix 1 Empirical Basis 213
Appendix 1.1 Requirements for Oral Proficiency of English Majors(Chinese version) 213
Appendix 1.2 Requirements for Oral Proficiency ofnon-English Majors(Chinese version) 213
Appendix 1.3 Requirements for Pronunciation of English Majors at Band 2 ＆ Band 4(Chinese version) 214
Appendix 1.4 Rating Scale ＆ Range Descriptions of CET-SET(Chinese version) 215
Appendix 1.5 Range Descriptions of TEM4-Oral(Chinese version) 216
Appendix 1.6 Range Descriptions of TEM8-Oral(Chinese version) 218
Appendix 2 Questionnaires 219
Appendix 2.1 OARS Trial Questionnaire 219
Appendix 2.2 OARS Teachers'Questionnaire 225
Appendix 2.3 OARS Students'Questionnaire 229
Appendix 2.4 Candidate's Questionnaire—the Pilot Study 234
Appendix 3 Experiments 238
Appendix 3.1 Experimental Test of Sample Rating Ⅰ 238
Appendix 3.2 Experimental Test of Sample Rating Ⅱ 242
Appendix 3.3 Experimental Test of the Validation Study—Role Cards 251
Appendix 4 Data of the Validation Study 254
Appendix 4.1 Candidates'Raw Scores on ORAL,TEM4 ＆ TEM8—the Pilot Study 254
Appendix 4.2 Raw Scores of Candidates'Ratings—the Pilot Study 255
Appendix 4.3 Raw Scores of Raters'Detailed Ratings—the Pilot Study 261
Appendix 4.4 Raw Scores of Teachers'Rough Ratings—the Pilot Study 265
Appendix 4.5 Sub-total Correlation Matrix—the Pilot Study 274
Appendix 4.6 Sub-total Correlation Matrix for English(key)Candidates—the Field Study 276
Appendix 4.7 Sub-total Correlation Matrix for Arts(key)Candidates—the Field Study 278
Appendix 4.8 Sub-total Correlation Matrix for Science(key)Candidates—the Field Study 280
Appendix 4.9 Sub-total Correlation Matrix for Arts(local)Candidates—the Field Study 282
Appendix 4.10 Sub-total Correlation Matrix for Science(local)Candidates—the Field Study 284
Appendix 4.11 Sub-total Correlation Matrix of RlT for 180 Candidates—the Field Study 286
Appendix 4.12 Sub-total Correlation Matrix of RAT for 180 Candidates—the Field Study 288
Figure 2.1 Allwright's diagram of communicative competence ＆ linguistic competence 18
Figure 2.2 Weir ＆ Bygate's framework of oral proficiency components 24
Figure 2.3 Rating scale of IELTS Oral 28
Figure 2.4 Oral rating scale of TEEP 32
Figure 2.5 Oral rating scale of American FSI interview 39
Figure 3.1 Inner structure of language competence 52
Figure 3.2 Trait factors of CLA 57
Figure 3.3 Cohen's rating scale for pragmatic speaking 58
Figure 3.4 Nunn's rating scale of turn-taking ＆ negotiation 60
Figure 3.5 Nunn's detailed rating scales of turn-taking ＆ negotiation 61
Figure 3.6 Nunn's rating scale of content of contributions 62
Figure 3.7 Nunn's detailed rating scales of content of contributions 62
Figure 3.8 Nunn's rating scale of pronunciation ＆ intonation 63
Figure 3.9 Nunn's detailed rating scales of pronunciation ＆ intonation 63
Figure 3.10 Nunn's rating scale of grammar ＆ vocabulary 64
Figure 3.11 Nunn's detailed rating scales of grammar ＆ vocabulary 64
Figure 4.1 Framework of OARS 96
Figure 4.2 Band descriptors of organizational competence 99
Figure 4.3 Band descriptors of pragmatic competence 102
Figure 4.4 Band descriptors of flexible interaction 104
Figure 4.5 Band descriptors of nonverbal communication 105
Figure 5.1 Histogram of RAT—the Pilot Study 135
Figure 6.1 Histogram of RlT for 180 candidates—the Field Study 159
Figure 6.2 Histogram of RAT for 180 candidates—the Field Study 159
Figure 7.1 Flowchart of designing OARS 183
Figure 7.2 Procedure of needs analysis in designing OARS 185
Table 2.1 Holistic oral rating scale of BEC 1 33
Table 2.2 Analytic oral rating scale of BEC 1 35
Table 2.3 Overall oral descriptors of BEC 1 37
Table 2.4 Rating scale of CET-SET 40
Table 3.1 Requirements for oral proficiency of English majors 45
Table 3.2 Requirements for oral proficiency of non-English majors 46
Table 3.3 Requirements for pronunciation of English majors at Band 2 ＆ Band 4 47
Table 3.4 Requirements for pronunciation of non-English majors 47
Table 3.5 Range descriptions of CET-SET 48
Table 3.6 Range descriptions of TEM4-Oral 49
Table 4.1 Demographic analysis of the trial questionnaire 76
Table 4.2 Analysis of the main component of the trial questionnaire 77
Table 4.3 Information of respondents to the OARS teachers'questionnaire 80
Table 4.4 Information of respondents to the OARS students'questionnaire 82
Table 4.5 Components of OARS—Teachers'perspective 87
Table 4.6 Components of OARS—Students'perspective 89
Table 4.7 t test on components of OARS(Teachers versus Students) 92
Table 4.8 t test on components of OARS(English majors versus non-English majors) 93
Table 4.9 t test on components of OARS(Arts majors versus Science majors) 94
Table 4.10 Reflection of language competence—sample rating Ⅰ 108
Table 4.11 Reflection of strategic competence—sample rating Ⅰ 109
Table 4.12 Reflection of scores—sample rating Ⅰ 113
Table 4.13 Reflection of language competence—sample rating Ⅱ 114
Table 4.14 Reflection of strategic competence—sample rating Ⅱ 116
Table 4.15 Retflection of scores—sample rating Ⅱ 121
Table 5.1 Statistical analysis of candidates'ratings—the Pilot Study 127
Table 5.2 Candidates'attitudes towards OARS—the Pilot Study 128
Table 5.3 Statistical analysis of raters'detailed ratings—the Pilot Study 129
Table 5.4 The correlation matrix of raters'detailed ratings—the Pilot Study 130
Table 5.5 Statistical analysis of teachers'rough ratings—the Pilot Study 131
Table 5.6 The correlation matrix of teachers'rough ratings—the Pilot Study 131
Table 5.7 Raw scores of RA—the Pilot Study 133
Table 5.8 Reliability estimates—the Pilot Study 136
Table 5.9 Descriptive statistics of the candidates'scores—the Pilot Study 136
Table 5.10 Sub-total correlation matrix—the Pilot Study 139
Table 5.11 Statistical analysis of outside measures—the Pilot Study 141
Table 5.12 Analysis of external validity—the Pilot Study 141
Table 6.1 Personal information of candidates—the Field Study 148
Table 6.2 Reliability estimates of six sets of scores—the Field Study 152
Table 6.3 Descriptive statistics of six sets of scores—the Field Study 153
Table 6.4 Analysis of concurrent validity coefficients for English(key)candidates—the Field Study 155
Table 6.5 Analysis of concurrent validity coefficients for Arts(key)candidates—the Field Study 156
Table 6.6 Analysis of concurrent validity coefficients for Science(key)candidates—the Field Study 156
Table 6.7 Analysis of concurrent validity coefficients for Arts(local)candidates—the Field Study 157
Table 6.8 Analysis of concurrent validity coefficients for Science(local)candidates—the Field Study 157
Table 6.9 Descriptive statistics of RlT ＆ RAT for 180 candidates—the Field Study 160
Table 6.10 Analysis of concurrent validity coefficients of RlT for 60 English majors—the Field Study 162
Table 6.11 Analysis of concurrent validity coefficients of RAT for 60 English majors—the Field Study 162
Table 6.12 Analysis of concurrent validity coefficients of RlT for 120 non-English majors—the Field Study 163
Table 6.13 Analysis of concurrent validity coefficients of RAT for 120 non-English majors—the Field Study 163
Table 6.14 Descriptive statistics of the candidates'average scores 164
Table 6.15 Frequency of candidates'performance on each trait 165
Table 6.16 MANOVA 1:Means across six groups 167
Table 6.17 Post-Hoc 1:Means across six groups 168
Table 6.18 MANOVA 2:Means across three fields 169
Table 6.19 Post-Hoc 2:Means across three fields 170
Table 6.20 MANOVA 3:Means across two university levels 171
Table 6.21 t test:Means across two university levels 171