Professor Kathi Bailey joins us to discuss teacher evaulation. Teacher evaluation can do so much good, but it can also end up doing even more harm. Professor Bailey tells us about what supervisors can do to earn trust, why supervision and evaluation is important, and how everyone can make the most out of the process.
So Professor Bailey, welcome to the podcast. You've got a quote at the beginning of your book about language teacher supervision which says that teacher evaluation is a theatre of the absurd and that teacher supervision can be seen as managing messes.
Why is that? Well, teacher evaluation is one of those fraught issues that's so important professionally and has such impact on individual teachers or may not have much impact on individual teachers. I think teacher evaluation can be a very chancy tool. It can be extremely useful and extremely dangerous and this is partly because it's such a complex issue.
When we talk about teacher evaluation we have to ask by what criteria are teachers evaluated and often, unfortunately, the criteria seem to be implicit and may be based entirely on the supervisor's or the administrator's view of what effective teaching is and may or may not have any relationship to what the research says about effective teaching. So I find the whole concept of teacher evaluation to be a really fascinating dilemma in our profession, one that can be used for good or ill or can be basically neutral or even non-existent. Yeah, and I suppose one of the issues with really coming up with any objective criteria based on research about what works is so much of language teaching is contextual and what works with, you know, for example, 80 students in a classroom in China is not going to work with, you know, 10 students in a classroom in the USA.
That's right. The context of teaching and learning is an important piece of how teachers should be evaluated and I think, too, the purpose of teaching and learning. We can have an overtly stated objective of a course to enable the students to communicate interactively in the target language, but how that objective is carried out varies greatly depending on, as you've just mentioned, the number of students in the class, the context in terms of how many hours per week of actual exposure to the target language do the students get, the teacher's own proficiency level and confidence in interacting in the target language, whether or not there are resources that allow the students to interact with others.
There's all kinds of stuff that goes into the contextual shaping of an evaluation process. So, to get back to evaluation, I guess at the beginning of evaluation we really need a description of what good teaching is, some sort of criteria. Given what we just said about the differences from one context to another, how can we ever develop criteria for assessing teachers? I mean, this is obviously something I guess that you've had to do yourself.
How have you gone about that, developing your own criteria? Well, as you just pointed out, the criteria are context sensitive or should be context sensitive and part of the factors that determine what we take into account have to do with the context. For example, I have visited on many occasions an English language school in Latin America where a certain teaching method was employed The teachers were trained in that method, they were to use that method and that was part of the evaluation was were they using the method. That made sense in that context.
If I were to take that criterion to another situation and say, okay, I'm observing Mr. Smith today or Ms. Smith today and she's not using that teaching method, it wouldn't make any sense if that teaching method weren't the policy adopted in the context, right? So, I think we have to look at the particular program in determining the criteria, maybe even the particular goals of the course. What are the objectives here? What are the teachers trying to accomplish? What are they supposed to be getting the students to be able to do? What level of aspiration are we working on here? So, for me, the criteria for evaluation have to change from one setting to another, sometimes from one class to another in the same program. And I would say even if you can be this flexible, even from one visit to another of the same teacher.
So, your question was how do I develop my own criteria? Well, unless I'm working as a consultant and I'm giving particular marching orders by the people that hired me in terms of observing, what I want to do is ask the teacher, what do you want me to look for? What's important for you today in terms of the kinds of things I might notice and be able to share? So, I want to start from the heart of the matter, which is the teaching learning context. What's happening with you and your students? What's most important? You have this, it could be seen as a burden, but it can also be seen as a luxury. You have this interested, impartial, concerned observer, and I would say in my case, a trained observer, who's come to spend some time with you.
How can we make the most of it? So, when I'm observing teachers, I start with what do you want me to watch for? Now, I guess one of the upshots of teachers being observed and evaluated is decisions get made based on those, right? That, I mean, for example, schools might have budget cuts and need to let some teachers go. What do you think about using evaluations to make sort of important decisions about teachers' lives? Yes, that's sadly right, and in fact, it's interesting that you mentioned this budget cut situation. I teach a seminar on language teacher supervision, and we were just discussing a case a few weeks ago where due to budget cuts and falling enrollments, some teachers have to be let go, and by what criteria do you make the choice of who to let go and who to keep on? It's a really difficult, real-world situation.
I think the first distinction to make is one that's often made, I think it comes to us from program evaluation and grant giving organizations initially, and that is, is the evaluation formative or summative? If it's formative, we're talking about, as you just referred to, Ross, we're talking about developmental purposes for evaluation. What can we do to help the teacher develop his or her skills, confidence, knowledge, etc., to the benefit of the program and of the students in the program? The other side, of course, is summative evaluation, which often refers to the idea that a decision rests on the evaluation. It can be a decision to retain or let go a teacher, a decision to promote or not promote a teacher, to provide tenure or not provide tenure.
These sort of important decisions that rest on evaluations are referred to as summative decisions, the products of summative evaluations. So it's important to me, working as a supervisor and doing teacher evaluations, whether in my own program or as a consultant in another program, to first understand, is this a developmental or an evaluative observation? And what decisions, what outcomes hinge upon this observation? I don't want to go into a classroom and pretend to be helping the teacher in his or her development when, in fact, I'm hired by the organization to decide whether or not he or she will stay. I want to be clear, is this a summative or a formative evaluation? At the same time, it's very, very sad to go into a classroom and be prepared to conduct a helpful developmental observation and find that teachers are terrified of being evaluated in a summative sense.
Yeah, I mean, do you think it's fair that teachers often feel like they're being judged whenever anyone walks into the classroom? I think teachers have every right to feel they're being judged. That's the predominant model. To me, the responsibility, many people will disagree with this and I say it unabashedly with no shame, the responsibility to be trustworthy lies with the supervisor.
I firmly believe that if I'm coming to watch a teacher and something hinges on that, let me contrast, I'm coming to watch a teacher for some sort of evaluation purposes, whether it's summative or formative, that's very different from me coming to watch a teacher because I want to learn what that teacher is doing so I can benefit myself. I think we often ignore the fact that observation could be extremely beneficial for the observer and I'm not just talking about the novice practice teacher who's doing his or her student teaching to attain a certificate or a credential or a degree of some sort. I benefit extraordinarily from every time I watch a teacher teach something I learn and that's a different benefit of observing than we achieve in the purpose of observing teachers for their benefit or their evaluation.
So I feel quite strongly that a supervisor has to earn a teacher's trust and I say this with all due respect to the kinds of teacher supervisors who, for example, may work for a ministry of education and have a huge geographical area in their country to cover that allows them to observe 20 minutes of a teacher's class once every three years and make a judgment on the basis of those 20 minutes as to whether or not those teachers are competent. That's an extraordinarily demanding job and one that I understand people must fulfill. It's not the job that I choose to do as a supervisor, unfortunately I've not been put in a position where I had to do that during my life.
So up until now when we've been talking about evaluation, we've been talking about that norm of an observer who walks in, observes a class and hopes that that's maybe representative of that teacher's practice. Can you tell us about the possibility of getting other people involved in teacher evaluation, for example peers, students themselves, even parents? How can some of those people be involved in the evaluation process? I think this is extremely important. There are so many people who are affected by teaching that it seems only sensible that they should have some say in how well that teaching is working.
And I want to emphasize teaching here instead of a teacher. So, for example, you just mentioned that students should have some say in the teacher evaluation process. This is called student evaluation of teaching.
In the literature it's often abbreviated as capital SET. A student evaluation of teaching is practiced in many countries, though not universally, and it tends to happen most with adult learners, either in university contexts or adult schools, community colleges, corporate organizations. It doesn't happen so much with children or, say, secondary school learners.
It also doesn't happen so much in contexts where it would be culturally inappropriate or odd for students to evaluate teachers. I understand that. I accept that.
But I do believe that students' evaluations are extremely important. Now, the research on student evaluation of teaching is mixed. There's a great deal of it.
It's fascinating. The major dilemma seems to be whether or not students can actually evaluate teachers' abilities on the constructs being measured or whether those evaluations amount simply to a popularity contest. Nevertheless, I think students' evaluations, where they fit culturally and are age appropriate, can be a really useful source of information.
For myself, I value peer observation a great deal. I've had some wonderful lessons learned from peers that I've worked with who've observed my teaching and helped me grow. And I also think that self-assessment is extraordinarily important in teacher evaluation and should have a place, whether it is through something as formal and as grand as compiling a teaching portfolio or something as localized and brief as writing a reflective commentary about a lesson that has been taught.
So I think peer evaluation, student evaluation, self-assessment by teachers are important pieces to include in teacher evaluation systems. So another way of thinking about evaluating how effective teachers are is just not to look at people at all, but just to look at students' test scores. Can you tell us a bit more about that and why teacher evaluation maybe isn't just as simple as seeing which teacher's students achieve the most or learn the most? So using student achievement measures as indices of teacher effectiveness is very important.
There's a history of research on that topic, and I find it quite informative. In recent years, that research paradigm has been given less attention for a number of reasons that made us call into question a somewhat simplistic assumption. And let me see if I can summarize the assumption.
In the research paradigm that uses student achievement as a criterion for evaluating teachers, the basic assumption is that good teachers' students will achieve more than poor teachers' students will achieve. It seems a very logical supposition, and it has, in fact, been used in a range of research. The difficulty is that it hinges on a couple of variables that may or may not be things we can count on.
So for example, the first issue is what is the measure of achievement? If students' achievement is being measured by a test, does that test adequately and appropriately reflect what was being taught? If not, the measure of students' achievement may not validly represent what the teacher was trying to teach. Another issue is whether or not the students in the particular course that's being evaluated through student achievement measures actually attained what they were supposed to attain in the time frame of the research. So if I'm teaching a semester-long course, and it's 15 weeks, and I meet my language students four hours a week, will they have accomplished at the end of those 15 weeks everything the course set out to teach them? Some will, some will exceed those expectations, and some won't meet those expectations.
So we have to realize that students are not uniformed, even when they are placed appropriately in a course according to some valid and reliable measure of their current knowledge, that students come into a class with great variation in their skills, in their knowledge, in their motivation, and so on. Before we use student achievement as a criterion for judging teachers, we have to understand that in any given class, you may have a wide range of abilities feeding into that evaluation. Now, another thing I would say is that the system may involve a kind of tracking in which highly motivated students, high achievers, are in one class, mid-range students are another, less proficient students are another.
If those three teachers are being evaluated by the same criterion, we clearly have different ability groupings going into the study. So I think student achievement is hugely important, perhaps especially when we have what's called a pre-test, post-test design, and that is where you have a situation where you know the student's ability before they take a class, and you assess their ability after they take a class, then you can see progress from time one to time two. We can look at how much students achieve from where they started to where they ended up without thinking, oh, they didn't up as high as we wanted them to.
Another piece of this, Ross, that I think is so important is that students learn at different paces, and so do teachers learn, so that what one student can achieve in a 10-week quarter or an eight-week program or a 15-week semester, another student may take more months to do, but eventually get there. So we have to recognize student achievement measures as time-bound and factor that into the use of student achievement data in evaluating teachers. Yeah, I feel in language teaching we commonly tend to think of language learning as taking place over a short period of time, like one lesson or one unit, but actually it's a really much longer-term process than that.
So I think we often end up with a lot of false positives, so students maybe who look like they've learned something because they said it at the end of the lesson or they wrote it down in a lesson, but those students might go on to forget that. And we also get false negatives, where students maybe didn't say something by the end of a lesson or didn't get something right on a test, but actually those students were also on the way towards acquiring that piece of language, but they just haven't fully learned it yet. I think that that's an extremely appropriate issue to keep in mind, whether we're talking about assessment for the assessment's sake or assessment for teacher evaluation, and that is that tests provide us with a snapshot of what a student can do on a particular day in a particular contest, whereas teaching and learning, they're more of a motion picture, possibly even a life history that takes place over time.
I wanted to ask you a bit more about observers giving feedback to teachers, because I think that's such a common part of teacher evaluation. In your book about teacher supervision, you mentioned giving a teacher feedback using what you call an I statement as opposed to a you statement. I think the example in the book is about a teacher who is giving instructions that are not very clear, but you use this way of giving feedback to sort of make this teacher aware of this issue, but doing it in a way that she can find kind of acceptable.
Can you tell us a bit more about that way of giving feedback? I remember that. I remember that lesson and that teacher. I remember being a little confused, and then later she realized that she might have been confusing in the way she gave the instructions.
It was a very powerful moment for me as a supervisor to stop thinking about what the teacher should be doing and to focus on what I was understanding. It put me in the role of the student for a brief time. I have to say, Russ, let me also tell you that in another chapter in that book, The Selfless Source, that David Noonan and Andy Curtis and I wrote, there's a chapter about video.
In there, there's a bit of a reflection about me watching a video of a workshop that I led, and I was observing myself. It was sometime after I'd given that workshop, but I wanted to have the experience of watching my own video. As I was watching myself and I was listening to the instructions I gave the group, I thought, what the heck did that mean? I mean, I even knew what was coming in the workshop, but the instructions I had just given didn't make sense.
Then somebody in the workshop said, what are we supposed to do here? I thought, oh, that's interesting validation that I myself am confused. One of the participants was a bit confused. What it means to use an I statement is to own the perspective, to talk about it from the perspective of what I, the observer or the learner or the peer, whatever, experienced instead of putting it on the teacher.
Instead of saying your instructions were confusing, I can readily say I was confused. There's a huge difference in the affect. The first one is an accusation.
Your instructions were confusing. The second is an admission. I was confused.
Those are two quite different speech acts, even though they refer to the same data. For me, as an observer of teachers and of learners, what I'm trying and have been trying for some years now to be more aware of and better informed about, more responsible for, is looking at my own reaction and analyzing it instead of just labeling a teacher or a teaching event. So another dilemma I think that often comes up for observers is deciding what to focus on after an observation.
Should you focus on what you, the observer, think should be highest priority for the teacher? Or should you kind of go with what the teacher themselves wants to talk about? At the beginning of the Self as a Source book, you have, I think, David Noonan talking about how supervision didn't work for him. Because when his supervisor came to observe him afterwards, David wanted feedback on his classroom management. But his supervisor kind of steered the conversation towards a focus on how he presented new language, which just wasn't really what he wanted to talk about in that moment.
Sure, I might have to make that decision on a case-by-case basis, but let me just take a slight digression and say I think there's a parallel here to language learning. Our syllabus may say that in the eighth week we cover relative clauses and we make sure that our English language learners can put the third person singular s on present tense verbs. When our students may be focusing on something else entirely, they may not be ready to worry about relative clauses or focus on third person singular s. What they want to do is get their kids to the doctors and understanding what the doctors and nurses are saying to them, that sort of thing, in a second language context.
So I think a key underlying issue here is learner readiness, whether the learner is a language learner or a teacher learner, a teacher, whatever. Who can pay attention to what at what time? So I think whether I'm a peer observer or a supervisory observer, I need to know what the teachers want to focus on. Honestly, I have to say this is like the first commandment of teacher observation.
What are you doing there? Why are you there? And why are you there could be I'm there to benefit. I want to learn about teaching or I'm there to help this teacher or I'm there to evaluate this teacher. But any of those have to include what is it that the teacher wants me to look for? What is it that I should be focusing on from the teacher's point of view? To me, this is absolutely bedrock essential.
And once I've done that, if I have met that responsibility, but I've also noticed something else. If I've built trust with the teacher that I will feel comfortable under time constraints, under the teacher's reactions to whatever we've just discussed so far saying, here's something I want you to think about. Or I also noticed the following.
I don't normally start out an observation report by telling the teacher what I saw and what he or she needs to know. I normally start out an observation, a post observation conference by asking the teacher where this conversation should go. But remember, I've started the observation by asking the teacher what do you want me to watch for? What do you want me to listen? I think there's a great deal of complexity that I'm skating over the top of at this moment.
But I really, really believe that awareness raising can be done through supervisors or other observers intervention, but only if the teacher is willing to listen. And a teacher won't be willing to listen if our, I have to admit, there are cultural ramifications of what I'm about to say. But a teacher can only listen if our message does not raise incredible defensive barriers.
So that's why I feel that the I statement is important. I was confused, or I was wondering, or I need to know, or I didn't understand. It's very different from saying your lesson was confusing.
It's a very different kind of statement. Isn't it interesting that sort of parallel can extend out into other parts of our lives of how we deal with our significant others, with our children, with our parents, with our friends. Telling somebody what to do is quite different from asking them about what they did and why, and sharing our views of what we saw them do and wondered about.
Absolutely. I mean, I think it's such an important skill, isn't it? Being able to give measured feedback. And as you say, it's something that people have to use or can use with their friends and loved ones.
And it's also hugely useful as a manager, even if you're not managing teachers. And yet, very, very few people that I know who've become supervisors of teachers really get any training on how to go about doing this. Yeah.
But Ross, did you ever know a six-year-old who looked up at you and said, Daddy or Mommy, when I grow up, I want to be a language teacher supervisor. I don't think I ever have heard that. No, I don't think so.
I think it's a part of our profession that many people fall into willingly or unwillingly. And being a supervisor, it's a skill set. It involves a skill set, one that I've been working on trying to develop since, oh Lord, since the early 1970s.
I've gotten a little better, but I have a long way to go. Well, finally, Cathy, thank you so much for joining us. Before we finish, where would you recommend that our listeners go apart from your great book, Teacher Supervision, a Case-Based Approach, to find out more about this topic? Ross, I'm so glad you asked me about teacher evaluation and teacher supervision.
This is one of my abiding interests, because I think supervisors can do a great deal of good and a great deal of harm, depending on how professional and appropriate their work is. If teachers or other listeners, supervisors, program administrators, policy makers, are particularly interested in this topic, I would like to refer them to the website of TIRF, T-I-R-F, the International Research Foundation. The full name of this foundation is the International Research Foundation for English Language Education, and the website is www.tirfonline.org. And the reason I bring it up in this context is that if you go to the homepage, you'll see a button about resources and references, and if you go there and then click on references, it'll take you to a webpage that has over 200 reference lists stored as Word documents that anyone can download and use for their own purposes.
And some of those reference lists have to do with what we've been discussing, for example, teacher evaluation, observation, and teacher supervision. So I would encourage any of your listeners who are interested in these topics to visit that part of TIRF's website.