Dina Edwards, an elementary principal in San Francisco, recently had a conversation with a 4th grade teacher following her observation of a math class this teacher conducted on prime numbers. For the first time in her 10-year principal career, Dina did not start the post-observation talk with "How do you think the lesson went?" Nor did she do most of the talking.
Instead, Dina relied on evidence from her 14-minute observation to pose an opening question that supported teacher reflection. She shared the observation evidence she had collected with the teacher (who we'll call Maya) prior to their meeting in the teacher's room for a post-observation conversation. Together, they analyzed the evidence, then Dina used coaching strategies to engage Maya in deciding on next steps for improvement.
Offering Evidence, Not Judgment
Dinais one of 150 principals participating in Project I4, an initiative funded by the U.S. Department of Education to improve academic discourse in STEM classrooms, and a project each of us is involved in implementing. As a part of the project, the Project I4 training team works with school leaders—principals, assistant principals, and district instructional coaches—guiding them to use evidence-based tools for classroom observations. In this case, Dina used a Project I4 tool to determine which equity strategies Maya used for calling on students. Dina took selective verbatim notes—a process for recording classroom observations for specific evidence. Then, before the post-observation conversation, she "coded" the notes, using a list of indicators from the tool to determine which specific calling-on strategies Maya had implemented in the lesson, and shared the evidence with her.
The evidence, as shown in Figure 1, indicated that Maya used hand-raising or cold calling as her primary calling-on strategies. Responding to learners who raise their hands and cold calling (saying a selected student's name, and generally providing no think time before that student responds) are practices that are prevalent in classrooms, but they don't support equitable dialogue (Hamilton, 2019). However, in this conversation, Dina didn't point a finger and tell Maya what to change. Rather, her opening question was: What do you observe about the method you used most frequently to call on students?
Figure 1. Sample Tally Sheet for Observation Focused on Calling-On Strategies
Making Classroom Observations Matter - table
Abbreviation | Calling-On Codes/Names of Practices for Calling on Students | Total Times Used |
|---|---|---|
| R | Raised hand | |
| CC | Cold Call (calling on student directly) | |
| CCD | Cold Call for Discipline (catching the student off guard) | |
| B-A | Blurt out-Teacher Accepts | |
| B-I | Blurt out-Teacher Ignores | |
| "C&R C&R/V" | "Call and Response (Teacher specifically asks for group response or indicates students should ""popcorn"") Virtual learning: Chat Box" | |
| ES | Uses Equity-based Strategy (e.g., equity stick or card to call on student) | |
| TR | Teacher Repeats student response to class verbatim | |
| TRV | Teacher ReVoices student response, paraphrasing the response for emphasis and often to form the next question | |
| TPS | "Think and Pair and then Share Virtual: Private Chat" | |
| Other | Any other strategy you note |
Leaders likeDina who are using Project I4 tools tell us that learning about effective observational practices has been a game changer. In her reflection at the end of the project, Dina asserted that, after years of supporting teachers, she finally feels she has a firm grasp on how to use evidence to approach observations and conversations more confidently.
Beyond Checklists
In many schools and districts, administrators use checklists to observe classrooms. After observing and checking off practices from a list, a school leader might leave a thank you note or provide brief feedback to the teacher, perhaps suggesting a ready-made strategy to try. But too often, observations, especially those connected to teacher evaluation, are what Toch and Rothman (2008) call "drive-by" observations that do not support teachers in changing instructional practices. These types of observations and feedback ignore the complexity of the classroom experience and the need for more nuanced evidence. Conclusions about observations are largely drawn from observer's subjective perceptions, which they use to give feedback to teachers with the expectation that they know what to do to improve. Such approaches are ineffective for changing teacher practice, and principals who continue to use them don't feel effective in improving teacher practice. Yet these practices persist in our schools.
ased on 50 years of research, the I4 observations emphasize collecting and analyzing evidence. The observation tools are customized to support principals so they can provide teachers with specific examples and objective evidence, and these tools address key components of equitable access and rigor. Equitable-access tools generate evidence about a teacher's calling on and questioning patterns; tools for rigor address the cognitive level of questions and how the teacher probes student thinking. Using the project's Effective Conversation Guide, I participants analyze videos of themselves having post-observation conversations after observing teachers' lessons; then they review their approaches with a coach and their peers.
We purposefully distinguish feedback from conversations. Feedback, although it can and should be useful for dialogue and reflection, is often unidirectional and relies on perceptual data. Instead, observers need to foster collegial conversations with teachers about changing practice.
To more effectively achieve its purposes, feedback needs to undergo both a quantitative and qualitative shift. School leaders need to think differently about the logistical aspects of a conversation (e.g., the tools we use and the questions we ask to promote teacher thinking) and about an invitational and collaborative tenor for conversations. These should be collegial learning experiences in which teachers make choices about their growth and development (Drago-Severson & Blum-DeStefano, 2017).
sing observation tools like those in Project I4 primes the pump for more effective post-observation interactions—and for changes in teaching practice. By encouraging school leaders to start with a smaller group of teachers before the observation tool is used widely, leaders build an initial group's capacity in using the key processes, and principals get a stronger grasp on their roles as instructional leaders.
Improving Academic Discourse
sing the Project I4 approach for observation and conversation entails three 15-minute segments: conducting the observation; analyzing the observation evidence and preparing an opening question; and facilitating a post-observation conversation. Leaders observe specific classroom practices focused on equitable access and improved rigor (Boykin & Noguera, 2011).
Leaders first use tools that support equitable access to have a full picture of the classroom discourse, then shift to more targeted tools as they have conversations with teachers. Teacher decisions drive the choice of tools to be used in subsequent observations. Teachers can also learn to use the tools themselves for peer observations.
The process is designed to be "chunked" and nimble enough to fit within an administrator's busy schedule. It aims to supplement, not replace, a district's formal evaluation requirements and procedures.
eaders use I4 observation tools to spotlight teacher actions that give learners equitable access, specifically the ways teachers call on students and how they form and ask questions. After teachers make progress on practices tied to access, leaders shift to tools that center on ensuring academic rigor, looking at a teacher's question levels, language-learning practices, and quality of dialogue. The observing leader collects objective evidence of exactly what the teacher and students do and say. As leaders become more proficient at selective verbatim notetaking, they time-stamp when a practice occurred. Then the leader codes or names specific components of the lesson and shares with the teacher—before the post-conference—both these selective verbatim notes and a tally of how often the various coded practices occurred in the segment observed.
The tally shown in Figure 1, for instance, shows calling-on strategies used in a short observation of a class with 18 students. Along with this tally, an observer would also use a seating chart and place marks next to the students called on. In this case, the observer's seating chart (not pictured) showed the teacher concentrated on only 10 students. Three students responded multiple times and eight students never spoke. Note that the teacher accepted one "blurt out" answer; it's not uncommon for teachers to accept blurt-outs and repeat responses, though they diminish student learning. Using such records of evidence, a principal could ask what patterns the teacher observes. Knowing the larger context of this classroom and the teacher's practice, the principal could pose coaching questions based on this data.
Converting Checklists into Stronger Tools
The kind of templates typically used for observations can be useful for naming or coding effective practices, but tend toward perception and judgment rather than offering evidence (Sullivan & Glanz, 2013). While designed for efficiency, checklists provide limited evidence of what the teacher actually did or said. Fortunately, principals, instructional coaches, and teacher leaders can learn how to convert checklists (and even evaluation rubrics) into evidence-based tools.
For example, using the academic discourse checklist shown in Figure 2, an observer would focus on checking boxes but often miss important evidence that the teacher needs to see precisely what she and students did or said in a part of a lesson. We would recommend changing a resource like this by eliminating the middle column with its "yes no, or N/A" response, which is perception based. By focusing on collecting evidence of teacher and student talk and then using the names of practice from the tool to tally for the teachers, the principal can have a more effective discussions with the teacher in the post-observation conversation.
Figure 2. Sample Observation Checklist, Unconverted
Note: This template is useful for naming or coding effective practices in teacher observation, but could tend toward recording an observer's perception/judgment rather than offering evidence, unless modified.
Source: Sullivan, S., & Glanz, J. (2013). Supervision that improves teaching and learning. Thousand Oaks, CA: Corwin. Used with permission.
Using specific terms or codes for practices gives a teacher better information. Consider the teacher indicators in Figure 2. Instead of "providing opportunities for students to speak …", a list of teacher questions can be coded or named according to levels of cognitive rigor using, for example, Bloom's revised taxonomy (remember, understand, apply, analyze, evaluate, and create) or Webb's (1997) depth of knowledge frame (recall, basic, strategic thinking, and extended thinking). The indicator "modeling reading processes like predicting" would be stronger if it included particular ways of reading: predicting, summarizing, or clarifying. And rather than using a general phrase like "listening carefully," observers list what they witnessed the teacher doing to listen carefully, for example, closer proximity to a student, making eye contact, and using student ideas to form the next question.
With such evidence, the teacher can make better choices about what to change. We urge leaders doing observations to convert checklists to evidence-based tools in this way.
Better Principal-Teacher Conversations
rief classroom visits of 12–15 minutes can yield enough evidence through coding and analyzing practice for a principal to have a meaningful conversation with a teacher. Project I4 participant Norman McDuffie, a middle school principal in rural North Carolina, told us he had tried for years to talk about the inequities in his classrooms—particularly how teachers didn't fully engage students of color in classroom conversations. For the first time in his administrative career, using these evidence-based observation tools and conversation process, he's now having conversations about equity issues he has observed in classrooms for some time, but hasn't had the evidence to point out to teachers.
In one instance, Norman observed a class of 28 students. The teacher called on students in the front of the class, using the typical practice of raised hands. This teacher—we'll call him Mark—responded to the same four students in the segment Norman observed, essentially ignoring the students of color and anyone else not in the two front rows.
Mark was surprised when he saw the evidence. Norman started the conversation by saying, "Let's focus on evidence I shared with you. What do you observe about the evidence?" Of course, Norman wanted Mark to focus on the need for more equitable participation and dialogue, but he wanted to hear how Mark would frame the issue. In their conversation, notice three characteristics essential to effective post-observation conversations:
1. The observer paraphrases what the teacher says as that teacher looks at the evidence. For example, the principal often uses "stems" like, "you seem to be noticing that ___" or "So, you are feeling ___" (identifying the essence of the teacher's remarks or emotions) or prompts that organize ideas, such as "There seem to be two issues here: ___ and ___."
2. The observer responses mention possible effective practices to try, as when Norman suggests that Mark have a student who doesn't know the answer call on another student.
3. The observer or coach relies on the teacher to decide what they will do to improve.
Mark: I see that I asked 23 questions while you were in the room and I usually called on only four students who raised their hands. I didn't realize I was doing that.Norman: You're noticing a pattern here: calling on only a few students. Let's look at this segment of the lesson in which you asked a question about a graph. For that one question, I noticed that 16 students raised hands to answer, and you called on Kara.Mark: Yes, it looks like I am only calling on the students right in front of me.Norman: There seem to be two issues here: calling on [only those with] raised hands and only calling on a few students. While calling on students with their hands raised is the most common practice in schools, we actually know that it's the least effective for student learning, which might be a surprise…. What other strategies have you used, or might you use, to get more students involved?Mark: Well, sometimes I have used [randomly drawing from] the popsicle sticks with names on them. But I had trouble with those.Norman: What was the problem?Mark: If I used the sticks to choose someone to call on and they didn't know the answer, then I just called on someone else who did.Norman: An idea for you might be: Pull a stick and, if the student whose name you pick doesn't have a response, then ask that student to repeat the question and to pick someone else using the equity stick jar. How might that feel to you?Mark: I could try that, but maybe I would be better off having them talk to each other first before I start asking questions. We had a workshop on think-pair-share, and I tried it for a while, but then I stopped.Norman: In other words, you're saying you might like to try think-pair-share?
Norman used paraphrasing techniques characteristic of good coaching that help the teacher in sense-making: acknowledging, clarifying, and organizing Mark's thinking. He inserted instructional information in the conversation—but as information, not feedback on Mark's performance. While the ratio of principal talk to teacher talk was about 50:50, teacher responses drove the conversation.
As a result of several observations and conversations with different teachers, Norman noticed a common pattern: inequities in calling on students. Norman decided to repeat the workshop on think-pair-share for any interested teachers. For the next month, he focused his observations of teachers on how to deepen the use of the practice in classrooms.
As a result of this professional learning and Norman's observations and conversations, teachers began to use think-pair-share and gained confidence in related strategies. For students, rehearsing in pairs generated confidence to speak in the large group. Teachers developed tools for assessing the use and quality of interactions using TPS and brought that data to weekly math department meetings, to continue to improve their practices.
From Knowing to Doing
In terms of urging teachers toward more equitable practices, many school leaders have a reservoir of will, but often lack the knowledge or skill to fully enact their equity principles. The tools and processes of this approach prime the pump for such leaders so they can effectively address equity—and spread practices that foster it.
Reflecting on many observations and conversations of school leaders participating in I has reinforced our belief that if school leaders focus on specific actions to observe teachers, analyze evidence, and engage in targeted conversations, leaders can support iterative changes in teacher practices—and feel they have a handle on being more effective instructional leaders, especially leaders for equity. Three components of the approach support this shift:
- School leaders have a strategic, do-able process for enacting instructional leadership, one they use to conduct frequent observations and conversations.
- Leaders shift from sharing with teachers mostly "big data" (like summative assessments) or "anecdotal data" to sharing specific evidence, providing teachers better information.
- All participating school leaders develop a common language for specific actions involved in instruction. This supports teachers and leaders in having individual and group conversations and developing personalized schoolwide professional learning.
Getting It Flowing
The purpose of priming a pump is to get water flowing. In this case, the water we want to flow is more equitable academic discourse. Many leaders we've guided know what they want to happen for students in schools. As they use the processes described here, we've seen them gain increased capacity and efficacy as instructional leaders—and get more equity flowing in their school.
Author's note: Project I is supported by a grant from the U.S. Department of Education, Supporting Effective Educator Development, Award No. U423A180096.
