Self-Assessment in Traditional Settings:An Asian perspectiveDr. Andrew Finch 1. Introduction As Gremmo observes (1995:151), the last 25 years have seen an increasing amount of attention to learner autonomy, self-directed learning, self-access systems and individualised/ independent learning in second language learning literature (e.g. Harding-Esch [Ed.] 1976; Altman & James [Eds.] 1980; Holec 1980; 1981; 1985; 1987; 1988; Geddes & Sturtridge 1982; Mason [Ed.] 1984; Riley 1985; 1988; 1996; Dickinson 1978; 1987; 1988; 1992; 1995; Wenden & Rubin [Eds.] 1987; Brookes & Grundy [Eds.] 1988; Ellis & Sinclair 1989; Little 1989; 1991; 1995; Sheerin 1989; 1991; Gathercole [Ed.] 1990; Wenden 1991; Page 1992; Esch 1994; 1996a; 1996b; Gardner & Miller [Eds.] 1996; Dam 1995; Dickinson & Wenden [Eds.] 1995; Pemberton et al. [Eds.] 1996; Benson & Voller [Eds.] 1997, Cotterall 2000). The general acceptance of these terms in the profession has prompted Little (1991:2) to describe autonomy as the ‘buzz-word’ of the 1990s, and Wenden to observe that “few teachers will disagree with the importance of helping language learners become more autonomous as learners” (1991:11). Brookes & Grundy (1988) see it as “axiomatic that learner autonomy should be the goal of every learner and every teacher” (1988:1), and Nunan (1996) claims that a degree of autonomy can be fostered in any learners and in any learning environment, though he stresses the need for learners to be “systematically educated in the skills and knowledge they will need in order to make informed choices about what they want to learn and how they want to learn” (Nunan 1996:13). Little observes that “genuinely successful learners have always been autonomous”, and that educators must “help more learners to succeed” rather than following learner autonomy as an explicit goal (1995:175). Various authors and researchers agree on self-assessment as a vital part of learner autonomy (Henner-Stanchina & Holec 1985:98; Dickinson 1987:16; Blanche 1988:75; Harris 1997:12), providing the opportunity for learners to assess their own progress and thus helping them to focus on their own learning. Hunt, Gow & Barnes even claim that without learner self-evaluation and self-assessment "there can be no real autonomy" (1989:207). Rea (1981) sees self-appraisal as helping the learner become aware of his/her responsibilities in planning, executing and monitoring his/her language learning activities, and Oscarsson agrees on this formative prime aim, adding a more summative secondary aim of enabling the learner "to assess his[1] total achievement at the end of a course or course unit." (1978). Dickinson points out that this does not necessarily devalue or conflict with external evaluation, which still has relevance for supplying official certification of learning (1987:136, cf. Van Lier 1996:120). Rather, as Dickinson & Carver observe: A language course can only deal with a small fraction of the foreign language; therefore one objective of language courses should be to teach learners how to carry on learning the language independently. Part of the training learners need for this purpose is training in self-assessment and self-monitoring. (Dickinson & Carver 1980) 2. Self-Assessment: history of research Self-assessment research in language education has had two main goals (cf. Heidt, 1979; Oscarsson, 1984): i) the investigation of possible ways of realising the goal of learner participation in matters of assessment and evaluation; ii) the investigation of the degree to which self-assessment instruments and procedures yield relevant and dependable results. In terms of applied research, Oscarsson has designed a number of simple self-assessment questionnaires, using behavioural specifications as a general frame of reference (Oscarsson 1978), and has further proposed a form of 'continuous self-assessment' as a possible model for an instrument intended to be used on a regular recurrent basis (Oscarsson 1984). Other practical and useful presentations of learner-centred evaluative materials and activities have been offered by Von Elek (1982 - a self-diagnostic test of Swedish as a second language), Lewkowicz & Moon (1985), LeBlanc & Painchaud (1985 - substitution of self-assessment questionnaires for the previously-used standardised proficiency tests in the university of Ottawa's second language programs), Heilenman (1991 - description of steps that may be taken in the practical development of self-assessment placement materials), Cram (1992 - practical illustration of self-assessment applied in the second language classroom.), and Harris (1997 - further examples of the role of self-assessment in formal settings). In terms of validation studies, investigation into the reliability and efficacy of self-assessment has shown an emerging pattern of consistent overall agreement between self-assessments and ratings based on a variety of external criteria (Raasch 1979; 1980; Palmer & Bachman 1981; Evers 1981; LeBlanc & Painchaud 1981; Achara 1980; von Elek 1981; Rea 1981; Blanche 1985; Oscarson 1978; 1980). Shrauger & Osberg (1981) claim that the relative accuracy of self-assessment is at least comparable to other assessment methods, and Oscarsson (1978) observes that adult learners studying EFL are able to make fairly accurate appraisals of their linguistic ability using a variety of scaled descriptions of performance as rating instruments. Wilson (1996), using English, German and French language self-assessment adaptations of the FSI/ILR[2] oral proficiency rating scale, in conjunction with an objective norm-referenced test, found that participants were capable of placing themselves "as they probably would have been placed, on the average, by professional raters using the (FSI-type) Language Proficiency Interview procedure" (Wilson 1996). Despite these findings, a number of pedagogic issues have arisen concerning the concept of self-assessment as an evaluation tool. Some of these are listed below, along with responses which have been made to them¡:
Oscarsson (1997) offers a "tentative" summary of the current state of self-assessment research relating to language education, warning that it would be premature to draw far-reaching conclusions about findings at this stage:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
3. Justifications The lack of research into self-assessment to date has meant that most justifications (as for autonomy in language learning) have been a mixture of the educational, humanistic, philosophical, sociological and pyschological. Dickinson (1987) invokes learning theory, claiming that "the ability to evaluate the effectiveness of one's own performance in a foreign language is an important skill in learning, and particularly important when the learning becomes autonomous." (Dickinson 1987:136; cf. Trim in Oscarsson, 1978:ix; Council of Europe document, 1974:7). Harris (1997) stresses the psychological benefits of self-assessment: "Above all, they [learners] can be helped to perceive their own progress and encouraged to see the value of what they are learning. … The best motive to learn is a perception of the value of the thing learned" (Harris (1997:19). Van Lier voices the humanist perspective: "In addition to 'normal' testing, we need to pay attention to the basic moral purpose of education: promoting the self-actualization of every learner, to the fullest" (Van Lier 1996:120), and Harris draws attention to the importance of affect: "If we attend to the affective and cognitive components of students’ attitudes … we may be able to increase the length of time students commit to language study and their chances of success in it” (Harris 1997:20). Dickinson associates self-assessment with the process paradigm in language teaching (Dickinson 1987:151, cf. Breen 1987), and a number of authors stress the learner-centred nature of self-assessment (Oscarsson 1978:1; Van Lier 1996:119; Harris 1997). Of particular significance for the present study, Harris (1997:19) sees self-assessment as a practical tool that should be integrated into everyday classroom activities, and Blanche proposes that self-appraisal "would be particularly helpful in the case of false beginners" (1988:85). Though much of the self-assessment debate focuses on its feasibility and practicality for self-directed individuals, often in self-access study situations, Harris (1997:13) also sees self-assessment as appropriate in test-driven secondary and tertiary education, claiming that it can help learners in such environments to become more active, to locate their own strengths and weaknesses, and to realise that they have the ultimate responsibility for learning. By encouraging individual reflection, "self-assessment can begin to make students see their learning in personal terms [and] can help learners get better marks." (Harris (1997:13). Peer assessment (a form of self-assessment [Tudor 1996:182] and justified largely by the same arguments) is especially applicable to the classroom setting, aiming to encourage students to take increased responsibility for their own curricula and to become active participants in the learning process (Hill 1994:214; Miller & Ng 1996:134). Tudor adds that critical reflection on the abilities of other learners with respect to a shared goal is a practical form of learner training which helps individuals to assess their own performance, and which reduces the stress of error correction through identifying them in others (Tudor 1996:182). Thus Assinder (1991:218-28) reports increased motivation, participation, real communication, in-depth understanding, commitment, confidence, meaningful practice and accuracy when students prepare and deliver learning tasks for each other. Haughton & Dickinson (1989) (cited in Miller & Ng 1996:135) found "a relatively high level of agreement between the peer assessments and the marks given by the lecturers" in their study of a collaborative post-writing assessment (cf. Fok 1981). Students were: i) able to assess their own work realistically (to a large extent), even though most felt inexperienced as testers (lack of reliability) and were not comfortable with being tested by classmates (fear of losing face); ii) they were sincere; iii) they demonstrated a similar level of assessment to that of the lecturers; iv) the scheme did not result in a lowering of standards; and v) the students benefited in their understanding of and attitude towards assessment by taking part in the study (Miller & Ng 1996:142). Peer assessment can be therefore be seen as an effective means of involving learners in formative self-assessment (Miller & Ng 1996:134), with the presence of an audience in general having a positive influence on performance (Lynch 1988). Lynch also makes the important observation that "tutors can differ widely in their response to assessment of the same oral presentation", and that "we need to experiment with peer-based evaluation … to complement conventional tutor- and self-based assessment" (Lynch 1988:124). 4. The situation Self- and peer-assessment in traditional language-learning settings is thus desirable and feasible for pedagogic, practical, and humanistic reasons, though as Rea comments, curricula in such test-driven settings often reflect a view of evaluation which is ill-informed and concerned mainly with exclusion (“gate-keeping”) and administration (“number-crunching”): Although we would agree that language is a complex behaviour and that we would generally accept a definition of overall language proficiency as the ability to function in a natural language situation, we still insist on, or let others impose on us, testing measures which assess language as an abstract array of discrete items, to be manipulated only in a mechanistic way. Such tests yield artificial, sterile and irrelevant types of items which have no relationship to the use of language in real life situations. (Rea 1978:51, cited in Weir 1998:3) If students are to learn in a way that motivates and is meaningful to them (given the literature suggesting that this will enhance and promote language acquisition), the learning environment needs to include consciousness-raising (language learning awareness), reflection (self-assessment), and development of learning strategies, as part of “actual” language study. Assessment in this context exists to give information to the learner and the teacher in terms of learning strengths and weaknesses, so that future goals can be set and learning plans devised. Testing which concentrates on the “target-like appearance of forms” (Larsen-Freeman 1997:155) ignores the fact that “we have no mechanism for deciding which of the phenomena described or reported to be carried out by the learner are in fact those that lead to language acquisition” (Seliger 1984:37), as well as the fact that the learner’s internal grammar is not a steady commodity and often deteriorates prior to internalising new content. Even if we could identify and measure all of the factors in second language acquisition, “we would still be unable to predict the outcome of their combination” (Larsen-Freeman 1997:157). The question therefore arises of how to incorporate self-assessment into traditional language learning situations (e.g. state secondary schools). Governments often advocate alternative methods of assessment, claiming to be making “efforts to assess students authentically and holistically” (Korean Ministry of Education Website 2001), but “when push comes to shove”, the need for a non-complex “objective” method of discriminating between students tends to over-ride such statements, and teachers find themselves back in the test-driven classroom, preparing students to be measured on language “usage” rather than “use” (Widdowson 1978). This is the situation facing the majority of language teachers in national schooling and in private language institutions in Asia. Given that methods of assessment in use serve mainly administrative rationales, how is the individual teacher to proceed? Can change really be implemented from the bottom up? On the one hand, it would be unwise for a number of reasons for the individual teacher (or school) to introduce alternative testing unilaterally and exclusively, on the grounds that government guidelines are being ignored by education authorities, and that it is the responsibility of the individual to take action in this matter. Parents would be the first to object to such action, since their children would be the ones who (whatever their improved learning awareness, responsibility, motivation and fluency) would not have the rote-learning banks of instantly reproducible information necessary for entrance to the “best” universities. On the other hand is the depressingly long wait for educational change to trickle down from above. Even if there are teachers and educational gurus on the relevant government executive committee, their findings will be filtered through various self-serving administrative channels, and will bear little resemblance to the original intention of the policy-makers by the time they get to the classroom. 5. An approach Perhaps there is a “Middle Way” for the class teacher, rather than the bleak prospect of “put up or shut up”. The gradual introduction of holistic practices such as self-assessment into a traditional learning environment need not be confrontational or subversive, since consciousness-raising, reflection, and development of learning strategies can be shown to enhance traditional study methods and goals within the framework of institutionalised definitions of achievement. As mentioned earlier, a number of authors have suggested ways of implementing self-assessment in such settings. This paper builds on those suggestions by suggesting that any learning environment can become reflective and can promote positive attitude change, and that this change of emphasis will make the classroom experience more enjoyable and fruitful for teachers and learners. Taking Brindley’s (1984a:77) learner-centred system (fig. 1) as a model of reflective investigation of objective/subjective needs, which can be entered at any point, and which can continue during (and after) the course, we gain some insight into processes involved, though it might seem at first sight that learner-initiated needs analysis, “Objective-setting in consultation”, and “Negotiation” are not practical in institutionalised settings where students attend not by choice and have no say in the curriculum. Figure 1: Elements of a learner-centred system (Brindley 1984a:77)
Students typically perform their own (unassisted) needs analyses and have perceptions of their learning abilities. Even the statement “I am not good at English” implies a level of awareness, evaluation, feedback, negotiation, and objective-setting on the part of the speaker, whether this is well-informed or based on prejudice and popular learning-myths, and whether it is used to enhance future learning or to justify its discontinuation (e.g.: “I cannot learn, so I will stop trying”). Thus to see the learner-centred approach as out-of-place in the test-driven classroom is to confuse traditional teaching pedagogy with the system it serves, for as Underhill observes: … doing the same things with a different awareness seems to make a bigger difference than doing different things with the same awareness. (Underhill 1989:260) Reid (1999) points to the responsibility of teachers to “provide the scaffolding for more effective and efficient learning” (Reid 1999:305; cf. Guild 1994) by raising student awareness of affect, and then listening to the students as they express their needs, beliefs and perceptions. Underhill sees this act of “really listening to the student and to the content of what he or she says” (1989:256) as having a dramatic effect on the learning atmosphere, since “our students don’t necessarily need reassurance, what they need is to be heard” (1989:256). Such a student-centred approach presupposes a learning climate of trust and clarity, which Legutke & Thomas (1991:64) see as an indispensable goal, governing teachers’ choices and preceding the learning process, though depending on that process for its practical realisation. Awareness of the need for this trusting learning climate is generally seen as more facilitating than innovative tasks, techniques, or principles, since “a learning environment conducive to growth includes an atmosphere of trust” (Legutke & Thomas 1991:43). O’Neill stresses the importance of doing “ordinary things” well (O’Neill 1991: 300-301), and Sano et al. (1984) claim that creative production is possible only in a “non-threatening environment” which encourages meaningful learning and the creative use of English. They see learning as dependent on: … warm-hearted interaction between teachers and learners, as well as among learners themselves. This friendly interaction is, in our opinion, the most essential factor in successful language learning. (Sano et al 1984:171) Thus students might be encouraged to openly discuss the situation in which they find themselves (cf. Appendix C), to reflect on their learning abilities (cf. Appendix B), learning preferences/styles (cf. Appendices D, E, F) and goals (cf. Appendices G, H), and to identify what they need to do in order to achieve those goals (cf. Appendix J). This could well mean continuing in the same vein as before, but with a different awareness, in which learner-training, autonomy, and self-assessment would be seen as means towards such goals - ways of satisfying externally-imposed criteria in the most effective and painless manner. 6. Practical applications A non-threatening, warm-hearted, learner-centred classroom is therefore an indispensable requirement for the fostering of the sort of positive attitude change that will motivate students to become responsible, self-directed and effective learners. When we investigate the learning environment in Asia further, however, we find that language teachers are often faced with government-sanctioned behaviouristic texts of excessive content (cf. Lee 1991:18; Li 1998:686), which treat language-learning as a branch of arithmetic (i.e. as a collection of formulae to be learned and applied), which subscribe to teacher-centred presentation of formal concepts (Breen 1987), and which set their sights on national examinations and therefore have no time for concepts such as promotion of autonomy, positive affect, learner training, cultural sensitivity/suitability and critical thinking. In such a context, teachers cannot be blamed for wondering how to promote a more meaningful view of education. Given the non-threatening learner-centred environment, however, required texts can at least be supplemented (if not replaced) by materials which are directed to the learner, promote language-learning awareness and learner-training, and encourage students to reflect on and to assess their own (and each other’s) achievement. From here it is a logical step for self- and peer-evaluation to become a natural part of the learning process, whatever the context. The remainder of this study suggests some materials that might be useful in this process. These materials were developed by and have been used by the author in a tertiary setting (Freshman University students) since 1997. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
6.1 Self-assessment questionnaire Appendix A shows a deficiency analysis (“My English Ability”) intended for students to carry out by themselves. If used as a pre-course and post-course assessment (with items appropriate to the topics being studied) this can give learners a useful indication of how they see their initial level and any subsequent improvement. For Asian students, who typically under-estimate their abilities, this can be a means of showing them that they have perceived progress. Further meaning and relevance can be given to this activity by asking the students to design their own questionnaire. 6.2 Class Information In Appendix B we see a pair-work activity in which grading and attendance policy is explained. Students dictate the missing information to each other, and thus internalise the “rules” of the class. As with the previous activity, this introduction to accepting responsibility for what goes on in the classroom can gain increased meaning if it is owned and developed by the students. Rather than describe existing rules, therefore, students can be invited to participate in the definition of appropriate learning behaviour, by devising their own “Learning Contract” (Appendix C). The author has found that the preparatory groundwork for this activity can be effectively provided by a pair-work “Marriage Contract”. One means of inviting students and teachers to actively re-assess their roles in the classroom is through the instructions which appear in the learning materials. In Appendices A to J, all these instructions are directed at the students, and there is no indication of any “teacher talk”. Students therefore need to read and understand the instructions in order to perform the activities. For the teacher, this practice can quickly indicate students or groups of students who need assistance, while those who can understand are allowed to perform the activity in their own way, to extract the learning opportunities that they choose (cf. Van Lier 2000) (including accessing the teacher), to move on to the next activity at their own speed, or to develop more complex activities as they wish. This allows students to learn what they feel is relevant, at the pace that suits them. The classroom in which all groups work on the same activity at the same time, and in which the teacher says “Communicate now”, “Stop communicating now”, “Listen to me”, and “Communicate again”, is not promoting learner autonomy, and can only lead to frustration and decreased motivation for learning. 6.3. Needs analysis Involving the student in the decision-making process, and raising awareness about language-learning, are powerful rationales for any learning materials. As with most of the activities presented here, the aim in Appendix D is to “start the wheels turning”, rather than to analyse immediate outcomes. This is encouraged here through an interactive questionnaire dealing with learning preferences. The suggestion that there are many different learning styles might in itself be a novel one for individual learners, and would hopefully stimulate reflection and discussion on the topic. Comparing perceptions with classmates also helps to place self-assessment in the context of one’s peers. A similar activity in Appendix E (a “True/false” activity surveying class learning styles) leads to a “follow-up” activity in Appendix F, with students (in groups or pairs) compiling reports based on the data they found. If each student is responsible for researching one or two questions from the original survey, then this further activity can offer an opportunity for cooperation and teamwork. Members of each group can then report to other groups, etc. Willis (1996) and Skehan (1998:129-3) each offer five principles for the implementation of a task-based approach. These provide input, use, reflection on the input and use, and some attention to affect, though largely ignoring socio-cultural aspects. If one were to propose similar defining principles for the design of self-assessment materials, these would need to refer to the components of Brindley’s (1984) model (fig. 1), in addition to focusing on trust, responsibility and interaction. Since Breen & Candlin (1980:95) first defined language learning as “a process which grows out of the interaction between learners, teachers, texts and activities”, and Allwright (1984), in his “interaction hypothesis”, suggested that interaction might even be the learning process, interaction has taken a defining role in language learning, a situation supported by Van Lier (1996), for whom learning–awareness, investment, practice, and commitment: … are engaged, sustained, and augmented by learners’ work of interacting with others … social interaction … (inter+action) … means being ‘busy with’ the language in one’s dealings with the world, with other people and human artefacts, and with everything, real or imagined, that links self and world. (Van Lier 1996:147) 6.4 Goal-setting and reflection Appendices G and H look at the issue of learning goals. As with learning styles, this topic is often ignored in state education in Asia. Not only do students have to pass national exams to enter university, but the subjects which they study once there are also dependant upon entrance-examination scores. Occupation is not always a matter of choice! Learning goals can therefore be general and ill-formed, leading to learning habits which show no particular sense of direction. Appendices G and H therefore attempt to “set the wheels in motion” once more, raising the question of preferences, and leading by implication to consideration of effective methods of achieving those goals. The interview-format in Appendix H allows students to interact and share ideas on a personal level, though responses could once more be the subject of a class report in a follow-up activity. 6.5. Study skills Reflection on study skills is an integral aspect of self-directed learning, since these are the means by which learning goals are achieved. Self-assessment thus includes evaluation of the effectiveness of such skills. As with the other materials suggested here, the activity in Appendix J is directed at the learner, and refers to the teacher as a resource-provider. In addition to dice, the resources in this case consist of action cards (not reproduced in Appendix J), offering solutions to the questions posed in the board game. Appendix K is an example of a self-referencing activity, in which the means and the end are the same. In this activity, study skills are enhanced through reading, memory (remembering the text on the wall of the classroom), dictation, listening, writing, and comprehension-checking. The fact that these actions are all carried out by peers makes them more meaningful. Problems in Transcription, pronunciation, delivery, listening abilities, etc. are recognised by classmates in the group, as they are found to impede communication (“What did you say?” “How do you spell it?” “One more time please”). Finally, the text is about memory and means of enhancing it – a problem that may well emerge from carrying out the activity. 6.6 Performance assessment Appendix L offers a self-assessment that focuses on the performance of the individual on a learning project. This type of reflective activity can be devised for any learning situation, and is an example of the sort of instrument that might be used in a more ongoing assessment approach – one in which the student is continually monitoring his/her progress. Formats promoting such reflection are the learning journal (Finch 1998) and the portfolio. In the learning journal/diary, students write about learning experiences and perceptions, often including notes or letters to their teacher. These communications can then grow to become a source of joint reflection on the learning process, a possibility made more feasible in these days of email. An extension of this idea is the learning cassette (Finch 2000c), on which students record oral assignments in addition to describing their feelings about learning, providing a picture of the student’s progress over a period of time. The same is also true of the learning portfolio, in which students enter examples of learning achievements over a period of time. The mixture of process and product in this approach not only shows students how they have progressed, but provides them with concrete evidence to show the next teacher, or the prospective employer. 7. Conclusions In the recent shift in educational theory from transmission of knowledge towards transformation of knowledge, and to integration of knowledge with existing personal constructs and meanings (Kohonen 1999:280), assessment has taken on new affective goals in which the personal growth of the learner is becoming increasingly important (Ranson 1994:116). Thus it is no longer defensible to use discrete-item testing of dubious constructs or to sample performance as a means of inferring underlying competence or abilities, if assessment is really concerned with providing information on learning. Instead, the need to understand performance itself and the processing (and affective) factors which influence it, suggest an integration of assessment and instruction. This implies a re-evaluation of the methods used in language testing research, “to illuminate all of these unresolved issues” (Weir 1998:9). Favourable correlations of self-rating scores and external test scores support the use of self-assessment in second language learning, and various suggestions have been made for its incorporation into the traditional learning situation, where a focus on a warm-hearted learning environment and learner-centred materials, can result in: i) promotion of learning (Chamot & O'Malley, 1994:119; Dickinson 1987:136); ii) raised level of awareness (Blanche 1988:75); iii) improved goal orientation (Oscarsson, 1978:2); iv) expansion of range of assessment (Blanche 1988:75); v) shared assessment burden (Dickinson 1987:136; Harris 1997:17); and vi) beneficial postcourse effects (Dickinson 1987:136). |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
[1] References to gender (e.g. “he”, “his”) are reproduced as in the original version. [2] Foreign Service Institute/Interagency Language Roundtable |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||