当前位置:首页 > 语言文字
CET4作文评分人混合型反馈的效果研究
CET4作文评分人混合型反馈的效果研究

CET4作文评分人混合型反馈的效果研究PDF电子书下载

语言文字

  • 电子书积分:10 积分如何计算积分?
  • 作 者:徐鹰著
  • 出 版 社:广州:华南理工大学出版社
  • 出版年份:2014
  • ISBN:9787562343608
  • 页数:225 页
图书介绍:本专著是对大学英语四级考级(CET4)作文评分周期内对评分人混合型反馈的有效性的系统性研究,旨在探讨评分混合型反馈能否帮助评分人获得评分技能的理论和实践问题。评分人反馈是评分人培训中常用的一种方法,是保证考试信效度的重要手段。本书不仅包括对分数的多层面RASCH模型分析结果,还增加专家评分人的给分理据。此外还采用了基于有声思维的质性研究手段,力求建立中国特定考试评分背时景下评分人的培训和反馈的新型模式
《CET4作文评分人混合型反馈的效果研究》目录

Chapter 1 Introduction 1

1.1 Background of the study 1

1.2 Purpose of the study and research questions 2

1.3 Significance of the study 3

1.3.1 Theoretical significance 3

1.3.2 Methodological significance 4

1.3.3 Practical significance 4

1.4 A note on terms 6

1.4.1 Mock essays 6

1.4.2 Rater effects/variability/biases/errors 6

1.4.3 Rating accuracy 6

1.4.4 Scoring/score validity 6

1.4.5 Scoring/rating expertise 7

1.4.6 Cognitive strategy 7

1.4.7 Meta-cognitive strategy 8

1.4.8 Rating session 8

1.4.9 Feedback iteration 8

1.4.1 0 Counter-balanced design 8

1.4.1 1 The correlation coefficient 8

1.4.1 2 The mixed feedback 8

1.5 The structure of chapters 9

Chapter 2 Literature review 10

2.1 Introduction 10

2.2 Rater variability as a serious threat to score validity in performance assessments 10

2.2.1 Models of performance assessment of writing 11

2.2.2 Factors that affect rater performance 14

2.2.3 Rater variability from the psychometric perspective 17

2.2.4 Rater variability from the hermeneutic perspective 22

2.2.5 Rater cognition studies 23

2.3 Rater training as a routine measure to control rater variability 38

2.3.1 Overview of procedures of rater training 39

2.3.2 Studies on rater training effectiveness 41

2.3.3 Critical issues in rater training 43

2.4 Rater feedback as an important part of training 45

2.4.1 Studies on rater feedback effectiveness 46

2.4.2 Some common problems 51

2.5 Methodological considerations 52

2.5.1 Content of the feedback 52

2.5.2 Delivery of the feedback 53

2.6 Summary 53

Chapter 3 Method 54

3.1 Introduction 54

3.2 Research Design 55

3.2.1 The mixed methods design of this study 55

3.2.2 The quasi-experimental nature of this study 56

3.2.3 Context of the study 56

3.2.4 Materials 62

3.2.5 Participants 66

3.2.6 Instruments 70

3.2.7 Pilot study 72

3.2.8 Procedures 78

3.3 Data analysis 80

3.3.1 MFRM analysis of ratings 80

3.3.2 Analysis of degree of certainty 84

3.3.3 Verbal protocol analysis 84

3.3.4 The questionnaire of raters'perceptions of mixed feedback 86

3.3.5 The questionnaire of raters'evaluation of feedback effectiveness 86

3.3.6 Interview with those who are negative with the feedback effectiveness 86

3.4 Summary 86

Chapter 4 Results 88

4.1 Findings for Research Question 1:The effect of the mixed feedback on rater variability 89

4.1.1 Findings for RQ1(1):The effect of the mixed feedback on rater severity 89

4.1.2 Findings for RQ1(2):The effect of the mixed feedback on rater internal consistency 97

4.1.3 Findings for RQ1(3):The effect of the mixed feedback on rater bias 101

4.1.4 Findings for RQ1(4):The effect of the mixed feedback on rater central tendency 104

4.1.5 Findings for RQ1(5):The effect of the mixed feedback on rater severity DRIFT 106

4.2 Findings for Research Question 2:The extent to which EG raters can incorporate the mixed feedback into rating behavior when it is provided repeatedly over 3 iterations 109

4.2.1 First round results 110

4.2.2 Second round results 111

4.3 Findings for Research Question 3:The effect of the mixed feedback on raters'degree of certainty for their ratings 112

4.3.1 First round results 112

4.3.2 Second round results 114

4.4 Findings for Research Question 4:The effect of the mixed feedback on raters'deci-sion-making 115

4.4.1 Findings for RQ4(1):The effect of the mixed feedback on raters'decision-making in terms of text features 116

4.4.2 Findings for RQ4(2):The effect of the mixed feedback on raters'decision-making in terms of information processing behaviors 129

4.5 Findings for Research Question 5:Raters'perceptions of the mixed feedback 139

4.5.1 First round results 139

4.5.2 Second round results 145

4.6 Summary 151

Chapter 5 Discussion 153

5.1 Discussion of Research Question 1:What effect does the mixed feedback have on rater variability? 153

5.1.1 Methodological considerations 153

5.1.2 Interpretation of the results 153

5.2 Discussion of Research Question 2:To what extent can EG raters incorporate the mixed feedback into rating behavior when it is provided repeatedly over 3 iterations? 158

5.2.1 Methodological considerations 158

5.2.2 Interpretation of the results 158

5.3 Discussion of Research Question 3:What effect does the mixed feedback have on raters'degree of certainty for their ratings? 160

5.3.1 Methodological considerations 160

5.3.2 Interpretation of the results 160

5.4 Discussion of Research Question 4:What effect does the mixed feedback have on raters'decision-making? 162

5.4.1 Methodological considerations 162

5.4.2 Interpretation of the results 163

5.5 Discussion of Research Question 5:What are raters'perceptions of such feedback? 170

5.5.1 Methodological considerations 170

5.5.2 Interpretation of the results 170

5.6 General discussion:Establish an internal link between findings 173

5.6.1 Linking findings of RQ1,RQ3 and RQ4 173

5.6.2 Linking findings of RQ1,RQ3,RQ4 and RQ5 176

5.7 Summary 181

Chapter 6 Conclusions 182

6.1 Summary of findings 182

6.2 Implications of the study 183

6.2.1 Theoretical implications 183

6.2.2 Methodological implications 184

6.2.3 Practical implications 184

6.3 Limitations of the study and direction for future research 185

References 189

Appendices 203

返回顶部