CHAPTER 8: PROGRAMME EVALUATION (Continued)
Previous pages: 1, 2, 3, 4, 5, 6

8.8 Research instrument 6: teacher interviews  
8.8.1 Description
Structured interview sessions with all teachers on the programme were conducted in semesters 1 & 2 of the 1999 academic year. These examined changing perceptions of the programme and its efficacy, using a 9-point research instrument (table A-62, below). Topics on this instrument reflected issues which had been identified in teachers' meetings, informal conversations with teachers, and reflective teacher-observations, as relevant to the programme. Results for this instrument can be found in section tables A-72-88, and discussion of these results can be found in section 8.8.3.

TABLE A-62: RESEARCH INSTRUMENT 6: TEACHER-INTERVIEW QUESTIONS. 

8.8.2 Instrument 6: results
A similar process to that observed in the student interviews (research instrument 5, section 8.7) occurred with the teacher-interviews (administered three times in 1999), in that initial "open" conversations were subsequently focused into 9 questions (table A-62, above) in the 2^nd & 3^rd sessions. These latter interviews kept to the discursive nature of the interchange however, since teacher perceptions were well articulated and very relevant to this study. In addition to the formative nature of this feedback (in terms of the programme), the significance of this instrument for this study is in the insights it offers into teacher perceptions, opinions, and attitude changes. The striking (and authentic) picture of the programme offered by these comments is categorised in tables A-73-88 according to the 9 questions in research instrument 6, along with other issues which came up in the interview sessions. Entries in the left column of tables A-73-88 indicate the length of time the respondent(s) had been working on the programme, and "W" indicates written responses from teachers who had left the Language Centre the previous semester (having participated in initial interview sessions), and who agreed to contribute via email.

Table A-72 shows a condensed version of teacher-responses to these 9 questions, using only categories of response that arose. Representative verbatim selections from all three interview sessions are presented in tables A-73-88, and have not been edited for "acceptability". Only repetitive remarks have been left out, and comments which point out weaknesses or problems have been reproduced almost in their entirety. Recorded interview sessions normally took place as part of the discussion and reflection that occurred after lesson observation, and because of this, the topic occasionally shifted to teaching methodology, as teachers felt the need to discuss further what had happened in the lesson. These parts of the interview have been omitted from this study.

8.8.3 Research instrument 6: discussion
8.8.3.1 Question 1: "Is the programme achieving its goals (student confidence, motivation, independence)?"
Table A-72 shows condensed versions of overall responses to this re-wording of research question 2 (section 1.2), and it can be seen that only one teacher identifies a problem in this area – with Sophomore students. Three of the total of 23 answers are qualified ("to a certain extent"), 12 agree "in general", and 7 are strongly affirmative. Looking at the responses in more detail (table A-73), it is notable that initial responses (after 5 & 7 weeks on the programme) are very positive:

Yea – in a big way. I think the students are more confident in their English, and more confident in life. ... Relative to students in other schools, it seems that the students at Andong are less intimidated using their English and seem better able to express their ambitions. ... It shows that they've got CMI. (table A- 73)

These views become less impressionistic and more specific (though not less positive) as teachers gain more experience of the programme (e.g. after one semester), and there is evidence of them reflecting on the issues involved. Concepts such as student confidence, motivation and independence figure prominently, and teachers speak of students being "always willing to speak English", in contrast to other teaching experiences in Korea: "and you don't see that in other places that I've taught" (table A-73). One notable comment is offered by "one of the doubters", who states that "I've seen the students achieve a lot of confidence" (table A-73). There is little focus on teaching methods or personalities here, and more talk of the programme achieving its goals: "The way the programme is organised, the content helps to achieve these goals anyway" (table A-73). Teachers seem to agree that CMI goals are being achieved, especially for Freshman students (though there are reservations in terms of Sophomore students), and a number of statements indicate that teachers subscribe to the value of CMI as teaching/learning goals. One teacher links the three in a causal relationship, with trust at the starting point:

I see trust in the students. The trust inspires confidence, the confidence inspires motivation, and the motivation inspires learning. (appendix A-73) 

Comments about lack of motivation in Sophomores, and about students being able to "slip through the cracks" were confirmation of problems which had been addressed, but to which solutions had not by that time been found. One of these was the problem of students returning from military service and entering the programme in its second or third year, without working through the Freshman book. These students typically showed signs of confusion and inability to communicate, though they were not willing to study with Freshmen students, at a level which would have been more appropriate and useful for them (they would have been outside of the group in this case – cf. section 8.7.2.8). Another problem was that students were allowed (by university policy) to continue onto the next part of the programme, irrespective of whether they had failed the previous part. Students entering the Junior year without having passed the Sophomore or Freshman year, however,  could not be expected to understand their role in the third year programme, and presented the programme designer with a dilemma:

They [Juniors] do it at their own level, but some of them are at such a low level, that they could have benefited from doing basic stuff like the Freshmen. (table A-86)

8.8.3.2 Research instrument 6, question 2: "How do you feel about these goals?"
Although having been appraised of the goals and methods of the programme in job-interviews and in pre-interview communications, a number of teachers were surprised to find that the programme was in fact enacting ideas which were very different from the sort of guiding principles that they had previously experienced in their work or their professional training. Question 2 examines this issue, and receives only positive responses (table A-72), from "essential, excellent" (11) to "very important" (4), "appropriate" (4) and "I like them" (2), indicating overwhelming approval of the core ideas, and awareness of the relevance of a holistic approach to language learning:

Definitely. They're getting an education in English and they're learning about themselves and other people and how to get the most out of life. (table A-74)

Those are the most important things for a second language. (table A-74)

In one way, these are the best goals for students who don't have time and money to get into lessons. (table A-74)

They're very important. I like them. ... These are more like life skills. (table A-74)

This approach is appropriate for ANU students. (table A-74)

8.8.3.3 Research instrument 6, question 3: "Is the programme helping the students to develop their oral skills in English?"This question (a reformulation of research question 3, section 1.2) voices a concern implicit in question 2 of this instrument: "Even if the students are improving in CMI (a measure of programme success), to what extent is this resulting in improved oral skills in the short term (the original reason for setting the programme), given that it is almost impossible to monitor these in the long term?" Once more, the majority of responses (table A-72) are affirmative, ranging from "absolutely, certainly, definitely" (11) to "Yes, in general" (9), "Yes, to a limited extent" (2), and "maintaining and realising existing skills" (1). Opinions expressed during the interviews (table A-75) were also very positive, with anecdotes about the ability and willingness of students to engage in conversation, and the importance of CMI in facilitating this change:

Absolutely. I see all sorts of hypothesis-testing. I see all kinds of interlanguages that have been built up, that have been elaborated, that have been reformed, through the whole process of speaking, taking risks in an environment where they're not punished for making mistakes. (table A-75)

As in other questions, there are reservations, which are valuable for the feedback they provide: "Yes, but pronunciation is a problem" (table A-75).

8.8.3.4 Research instrument 6, question 4: "Have you noticed any attitude changes in the students during the programme?"The first three questions in this instrument examine teacher opinions regarding the effectiveness of the conversation programme. Questions 4 & 5 (cf. research question 1, section 1.2) introduce the issue of attitude change, a critical factor for the achievement of CMI. Responses by the teachers in table A-72 are more reserved than for earlier questions: "Yes. Positive/extreme changes" (7); "Yes, in general" (12); "Yes, to a certain extent" (3); "Yes, with some negative aspects" (3). However, the lack of negative remarks is again noticeable, and the opinions expressed in table A-76 overwhelmingly point to positive attitude change in students as perceived by teachers:

I've seen noticeable increases in their willingness to have conversations in English, and some classes have gone from not being particularly interested ... to being pretty hungry for it. Now they always come to class on time and are very enthusiastic and are rightfully proud of their accomplishments. (table A-76)

Not every student, but ... examples of students have taken off. At first they came in a little unsure of what they were doing, and by the end of the term they were self-correcting, and telling their classmates who were not working as hard as they should be to ‘let's finish this ... let's move on' and so I really was quite surprised. I wasn't expecting so much ... peer moderation. (table A-76)

[In the class I had for a whole year] they think themselves much better [with an] average 20% increase on the self-assessment instrument. In that class there are some attitude changes for the better. (table A-76)

Most of the [Junior] students do have good attitudes now ... Generally, they enjoy the projects. (table A-76)

Some classes didn't work co-operatively as much as they did before. I was a little worried, but actually they were learning to be a little bit more independent. (table A-76)

Yes. After several classes, a few students start to accept more responsibility [and] begin activities and "the class" without instructor prompting. Eventually, more and more of the students come to understand and accept this new responsibility in their English classroom. (table A-76)

I noticed an extreme attitude change in some of my classes from the first semester to the second semester. (table A-76)

It is noticeable that teachers identify positive attitude change, in contrast to their students: "They don't trust themselves, ... but certainly I can see the difference" (table A-76). While the programme can be said to have succeeded in the sense that teachers are identifying positive trends in the students, it is important that the students perceive these changes as well (cf. research instruments 2 and 5, sections 8.4, and 8.7) and such results point to the need for greater communication between teachers and students on this topic.

8.8.3.5 Research instrument 6, question 5: "Have you noticed any attitude changes in yourself during the programme?"Table A-72 shows 9 teachers recording definite, though difficult changes, 10 agreeing "in general" that attitude change has occurred, and 5 concurring "to a certain extent".  In addition to the "learning curve" that seems evident ("I'm going through such an incredible learning process", table A-77), a number of teachers also attested to increased CMI and improved affect in themselves:

 Yea. My confidence in the programme and my confidence as a teacher have increased. (table A-77)

 Yes. From the first semester to the second. ... At first, I wasn't sure of myself ... however, the more I just let go and relaxed ... I think now I'm feeling pretty confident. (table A-77)

 Once I got used to it, I suppose I was more motivated, once I started to understand what the programme was. (table A-77)

 I started to relax. I enjoyed the classes a lot more. (table A-77)

 I do trust the students a little more than I did at the beginning. (table A-77)

 There is no longer [any] fear of periods of uncertainty. ... When I first came here, I felt the need to correct. ... I'm less threatened. ... I see the purpose of having the students fill in the gap themselves.  ... I think it has given me more motivation and independence and confidence. (table A-77)

Responses to question 5 (table A-77) show evidence that working on the programme was in itself a learning experience for teachers, prompting considerable attitude change, which self-labelled "doubters" were not afraid to acknowledge:

 I've learned a lot from the programme. ... I was a bit cynical ... but when I came here I saw it could work ... I thought if the lid was taken off they would mess around. (table A-77)

 There has been an amazing growth in the students, semester to semester. ... Maybe my vision has changed.(table A-77)

I think my role as a teacher has certainly changed from controller to the other side of the spectrum. (table A-77)

I've finally learned that I don't have to explain everything. (table A-77)

It's been a positive change. ... I've been teaching a number of years by the traditional method, and it was really difficult for me to get out of that mode. (table A-77)

At first I didn't really buy into your ideas ... This year I am really happy with the programme. (table A-77)

I, at first, was quite discouraged by the format, as it was in opposition to my own training. ... By a format which was mostly trial and error, I learned to adjust to the new paradigm. It was wearying, and sometimes demotivating, but finally yielded considerably positive results – both in my students and in myself. (table A-77)

Yes. Moving away from a teacher-centred, teacher-controlled learning environment, has required me to become more patient.  Fighting the temptation to dominate the direction and "learning" content and pace, took some practice, to be honest. In classes with students who are used to and more than willing to give all control to an instructor, I had to struggle to change my attitude about my role in the classroom. (table A-77)

It really changed my role in the classroom. It made me more interactive and ... a much more humane type of a teacher. Broke down the old role models. (table A-77)

There were two instances of teachers who discovered that they didn't in fact believe in the student-centred holistic approach, and that they were most comfortable teaching grammar in a "3-Ps" format. These teachers were not able to agree with core ideas of the programme, and left at the end of their contract. Such instances point to an important role played by the programme - helping teachers to identify their real beliefs, and to reflect on their teaching approach: "I hadn't actually thought much about my own goals as a teacher before I came here" (table A-74).