A Reader Asks... The 13 Question Final Exam

A LYS campus instructional leader calls me out.

SC,

A 13-question final better than what we already have?  Empty boast or the real deal?

SC Response

My goals with a comprehensive final are to:

1. Assess student mastery of the content.

2. Determine if the scope of the content was covered.

3. Determine which concepts we, as a content team, teach effectively

4. Determine which concepts we, as a content team, do not teach effectively.

To do this I need to have questions that assess the critical elements of the content, and I most likely want to have multiple questions for those elements.  Which means that for a given course, I will have between 25 to 50 questions that I want to ask.  And you can ask all of those questions.

You just don’t have to ask each one to every student.

Take your question bank, ensure that the questions for each element are of similar difficulty and then make multiple versions of the test.  If I had a 30-question bank, I would create three different 10-question tests.  The student would get his individual score, and I would aggregate the item results of the three tests for my instructional planning data.

Now I said a 13-question test.  We all agree that multiple choice tests aren’t the best way to assess student learning.  So I would have 3 essay questions that allow students to truly demonstrate the depth of their knowledge of the material.  And I would weigh the final, 75% essay, 25% multiple choice.

With this format, a 90 minute, 13-question final will provide the richest and deepest sample of student and instructional data you have ever possessed.

The real questions are, “Do you want it?”

Or, “Is doing the same old thing and not knowing, better?” 

Think. Work. Achieve.

Your turn...

• Call Jo at (832) 477-LEAD to order your campus set of “The Fundamental 5: The Formula for Quality Instruction.” Individual copies available on Amazon.com!  http://tinyurl.com/Fundamental5 
• Now at the Apple App Store: Fun 5 Timer (Fundamental 5 Delivery Tool); PowerWalks CLC (Networked Formative Observation Tool) 
• Upcoming Presentations: TMSA Winter Conference; ASCD Annual Conference; TEPSA Summer Conference 
• Follow Sean Cain and LYS on www.Twitter.com/LYSNation  and like Lead Your School on Facebook

The Power of Instructional Coaching - Round 1

A LYS Assistant Principal writes:

SC,

We conducted our first round of PowerWalks conversations with our teachers in late October and our biggest takeaway from these initial conversations was the lack of critical writing taking place on our campus. We couldn’t hide from the data; in our classrooms critical writing activities were an infrequent occurrence. 

But a direct consequence of these conversations is that we are now seeing critical writing activities in the classrooms of more than half of our teachers.  Beyond this positive note, the administrative team and a great many teachers have expressed how much they enjoyed the opportunity to talk shop for a few minutes.  I believe teachers are beginning to value the insight the data is providing them. And I can report that the administrative team enjoys some of the certainty the data gives us as we coach for the purpose of improving the quality of classroom instruction. 

SC Response

Fantastic news! Though not surprising considering the focus of your Administrative Team and the passion of your Teachers.  I look forward to your field report on the second round of PowerWalks instructional coaching conversations.

Think. Work. Achieve.

Your turn...

• Call Jo at (832) 477-LEAD to order your campus set of “The Fundamental 5: The Formula for Quality Instruction.” Individual copies available on Amazon.com!  http://tinyurl.com/Fundamental5
• Call Jo at (832) 477-LEAD to order your campus set of “Look at Me: A Cautionary School Leadership Tale” Individual copies available on Amazon.com!  http://tinyurl.com/lookatmebook 
• Now at the Apple App Store: Fun 5 Plans (Fundamental 5 Lesson Plan Tool); PW Lite (Basic PowerWalks Tool); PW Pro (Mid-level PowerWalks Tool)
• Upcoming Presentations: NASSP National Conference; The 21^st Century High School Conference 
• Follow Sean Cain and LYS on www.Twitter.com/LYSNation  and like Lead Your School on Facebook

A Reader Writes... (Student vs. Teacher Data - Part 1)

In response to the 8/17/2011 post, “Student vs. Teacher Data,” a LYS Principal writes:

SC,

I enjoyed your post, “Student vs. Teacher Data.” Ronald Thomas shares your thoughts as he writes the following on this very subject in Education Week (My Nine Truths of Data Analysis - 6/15/2011):

"Focus on instructional insights derived from data. We don’t need ‘data driven’ schools. We desperately need ‘knowledge driven’ schools…Data are useless unless organized into meaningful patterns. Some schools are getting better at creating charts and graphs that can be useful to teacher teams, but others are drowning in data. The most important questions in data analyses are not ‘What did the students score,’ and ‘How many passed”. The most important questions are:

• What do the students know?
• What do they not know?
• What are we going to do about it?"

Common assessments, utilized properly, target short-term lesson planning and drive the planning/instructional delivery engine. Without adjusting lesson plans and adult behavior according to the teacher data—we are simply wasting a tremendous amount of time, energy, and printer ink.

Lead On.

SC Response

I take Thomas’ three questions to an even more focused level.

1. What practices improved student performance?
2. What practices stymied student performance?
3. How do we know this?
4. How do more or less of the identified practices.

When a campus is asking and answering these questions within short assessment windows, the effect is continuous incremental improvements to schools systems and teacher practices that result in increasing student success.

The leadership challenge is to push and pull staff up to the point where they see, experience and appreciate the value of working in such a system. This does not magically occur. But once it does (as you are experiencing at your multi-year, exemplary campus) just point them in the right direction and get out of their way.

Think. Work. Achieve.

Your turn...

Call Jo at (832) 477-LEAD to order your campus set of “The Fundamental 5: The Formula for Quality Instruction.” Individual copies available on Amazon.com! http://tinyurl.com/4ydqd4t

Attend the LYS presentations at the Texas School Improvement Conference on 10/27/11

• Improve Now – 8:00am
• The Fundamental 5 – 9:45am
• You Can’t Fix What You Can’t See – 10:15am

Follow Sean Cain and LYS on www.Twitter.com/LYSNation

Confirmed 2012 Presentations: NASSP Conference; NASB Conference

Student vs. Teacher Data

Student vs. Teacher data: I recently had a frustrating meeting about this subject with someone who should know better (a big district C & I director). Parroting E. Don Brown (which is always a good idea), “We have beat student data to death. Let it go.”

I’ll paraphrase the meeting

1. An aligned curriculum should have short-term common formative assessments and a summative assessments.

So far so good, we’re on the same page.

2. No class (excluding severe need special education) gets to opt out of any of the assessments.

Now they’re concerned. “Aren’t we limiting our really smart students by making them take dumbed down pedestrian tests?” No, and now I’m concerned.

3. The short-term common assessments are not used to identify which students need remediation. They are used to identify which instructional strategies work best with our students and which teachers are implementing those strategies.

Now I’ve lost them. “Are you saying we shouldn’t tutor?” No, I’m saying that it is no longer an option to continue to teach ineffectively.

4. The annual summative test (TAKS / STAAR in Texas) is a program review. Did we effectively teach what we were supposed to teach, or not. We use results to identify the big holes in our curriculum and our delivery; then we change and revise our practice to fill those holes in the subsequent year.

Now they smile. “We do that, we schedule our TAKS failures into TAKS remediation or double block content classes for the next school year.” Now I frown. “That’s not what I said.”

Assessment identifies whether or not our practice is effective. Student performance is merely the consequence of our action. To continue to focus on student interventions is to focus on the symptom and not the cause. Treating the symptom does provide some immediate, short-term relief. Treat the cause and at some point the symptom disappears. Shift the focus of data from students to adults and we quickly transform from being the victims of fate (those kids, those parents, etc.) to the masters of our destinies (our work, our knowledge, or expertise).

Think. Work. Achieve.

Your turn...

Call Jo at (832) 477-LEAD to order your campus set of “The Fundamental 5: The Formula for Quality Instruction.” Individual copies available on Amazon.com! http://tinyurl.com/4ydqd4t

Follow Sean Cain and LYS on www.Twitter.com/LYSNation

More What Does the Data Show

Recently, I was working with a new LYS High School Principal. We have just completed our first semester supporting her campus and we were reviewing mid-year progress. One piece of data that the principal was excited about was the reduction in the number of teachers who had failure rates greater than 20%. In the Fall of 2009, the campus had 11 teachers with a first semester failure rate greater than 20%. In the Fall of 2010, that number had dropped to just six teachers.

That is a dramatic 45% improvement!

But it begs the question why and how? So we dug a little deeper. On this campus, there has been a focus on improving the implementation of the Fundamental Five. So we created a PowerWalks report that compared the frequency of the Fundamental Five in the classrooms of the high failure rate teachers against the rest of the campus. What the principal was able to see was that the six struggling teachers are not miles behind the rest of the campus; instead they are 6% to 11% behind the campus. This is important for three reasons. First, it means that the six teachers are not hopelessly out of touch. And in fact, with a little extra direct support, they should be able to quickly close the gaps between themselves and their peers. Second, it illustrates how sensitive students are to even minute changes in adult practice. Finally, it drives home the understanding that student failure has less to do with teacher expectation and more to do with the quality of instruction.

How can you not love data?

Think. Work. Achieve.

Your turn...

Visit the LYS Booth at the TASA Mid-Winter Conference

A Reader Asks... PowerWalks

A regular LYS reader asks,

“I'm very interested in PowerWalks. Our district utilizes the 3 minute walk through, which takes longer than 3 minutes, that's for sure. Not sure that I'm 100% committed to this way of doing walk throughs. I'm looking for something that will benefit my teachers and students better."

SC Response
Some quick background information. There are six basic categories of classroom observations. However, as you will see, each specific protocol will overlap into multiple categories. Briefly the categories are as follows:

Extended time observations
Longer than 15 minutes. These observations are usually scheduled and provide the teacher the opprotunity to demonstrate all that they know (the dog and pony show). But the dog and pony show is important. It allows to observer to gauge the current ceiling of teacher performance.

Short time observations
Less than 6 minutes. These observations are usually unannounced and give the observer an opportunity to gauge a teacher’s typical practice.

Subjective observations
Requires the observer to make judgment calls on quality and/or teacher intent.

Objective observations
Requires the observer to monitor only specific and observable practices and activities.

Formative observations
Collected data and observations are for the purpose of coaching teachers and improving performance

Summative observations
Collected data and observations are for the purpose of evaluating or ranking teachers.

All of these observations are useful, but as you can see the categories serve different purposes. Where administrators get in trouble is when they use the wrong tool for the task. Or they try to make the “Swiss Army knife" observation protocol. And like the Swiss Army knife, though it may look neat, it darn sure isn’t useful.

So now we get to practice. You need two tools. One is an extended time observation tool, primarily used in the summative evaluation process. The other is a short time observation tool, used only for formative assessment. Anyone who tries to convince you that one tool can do both is either trying to sell you something, is lazy, ignorant, and/or dangerous (I’m begging, someone try to debate me on this). Also, stay away from subjective tools. What you think is much less valid than what you actually see. You will never build sustainable system-wide capacity through subjective means. You may be able to develop individual talent, but if you are relying on individual talent to sustain your organization then you are already dying a slow death.

So where does PowerWalks come into play. It is a short time, objective, formative classroom observation tool. Nothing unusual there. But it has the most powerful disaggregation tool available attached to it. It constantly evolves and it is inexpensive. Nothing comes closer to giving you and your staff “game film” of instructional craft. Anything more would be a sales pitch, which I won’t bore you with. But if anyone in the LYS Nation is interested in trying the tool, you can use it risk free for a semester. And if you are attending the TASSP Summer Conference, every attendee gets PowerWalks for their campus for a semester for free. Plus, TASSP will train you how to use it, at the conference.

Think. Work. Achieve.

Your turn...

A Reader Writes... (Working Hard - Part 2)

In response to the post, “Working Hard,” a reader writes:

Thanks for getting to the core of the supposed 'low morale' issue that always occurs at this time of year. Teachers are aware the clock is ticking. And the ones that 'overslept' one day too many know that their data doesn't look so hot right now, just like I do. This means that I’m on their doorstep everyday with a big bright smile and “You better believe I’m watching you like a hawk” eyes.

They also dread hearing my annual comment of “You better not even make a prediction right now of end of year results for ANY student because we still have three months of teaching ahead of us!"

You've said it before and I hope they carve it into all LYS’er tombstones... 'Adult behavior determines student achievement'.

Cowgirl up!!!! :)

SC Response
There you go. But for the record, you did misquote me. It is, “Adult practice drives student performance.”

I do want to point out that you made an excellent point. We have three months left of school. This means we have to keep teaching after the state accountability tests have been administered. Unfortunately, this is rarely the case; most schools shut it down and coast after the tests. Here’s the litmus test to see if this is occurring on your campus. If you are performing at the same level (or lower) as your demographic peers, there is a lot of phoning it in occurring. If you are out performing your demographic peers, you are most likely teaching at full speed for most of the year.

Numbers don’t lie.

Think. Work. Achieve.

Your turn...

Data Use

There are two basic types of data to work with in a school, student data and teacher data. Using student data is the first stage of data analysis and if you aren’t using data on your campus this is where to start. Student data will indicate where the low hanging fruit can be found. For example, I was working with a campus and in the midst of doing some item analysis, we discovered that students were overwhelmingly missing science questions that dealt with application. That problem was solved by getting students in the lab for more hands-on activities.

However, student data will only take you so far. The next stage of data use revolves around teacher data. It is the team analysis of this data that is the foundation of a truly vibrant professional learning community. To begin this process, leadership must provide teachers with three tools. The first is a common scope and sequence, the second is short-term, common assessments and the third is hyper-monitoring data. These three tools allow teachers to identify which teachers make the biggest and most consistent gains with the campus’ most academically fragile students and which components of pedagogy seems to make the biggest impact in the classrooms.

Armed with this information and time to plan, learn and adapt, an instructional staff can go from sub-par to extraordinary in less than a year.

Your turn…