A Reader Writes... (Common Assessments for Students with Special Needs)

In response to the 12/9/2010 post, “Common Assessments for Students with Special Needs,” a LYS Principal writes:

OK, the confusion and misapplication is amazing.

First, everything the special education director said is 100% accurate. I had this same discussion with my special education director and made the decision to allow accommodations/modifications to special education student scores but to NOT count them in my common assessment data analysis. I did this because we count the common assessments as a grade. I have done it both ways. That is, I have counted common assessments as grades, and I have not. I don't have an opinion on which way is best, but if you count them as grades follow the special education director's advice.

Second, I once wrote on this blog to not fixate on 70 when determining passing rates. In response I have had people tell me, "We don't use 70%, we use 80%". I have news for you - that is not better. It is likely that is worse.

I possess a gift that shines above that of most other people. I understand math at a very deep level. I was a physicist and research scientist for a major university before entering public education. With that fact behind us, let's get this one out of the way - your assessment is likely "invalid" in the statistical sense. That doesn't mean it is useless. "Invalid" used in this context is a technical term, not a judgment.

The next thing to get out of the way is that once you finally decide to accept the fact that your assessments are invalid, you can then move on to the fact that it is virtually impossible to compare performance on one invalid assessment to another invalid assessment.

Many at this point, then ask, “Why bother?” The answer is simple. Use your common assessments to compare the performance of one group against a different group on the SAME exam. Even if the exam is invalid (yours are, accept it), it is still reasonable to look at the performance of two different groups on the SAME assessment.

Example: Your high SES kids are scoring 90% or better on all levels of TAKS. Your low SES kids are performing 45% on average on TAKS. You have a 45% achievement gap on a VALID assessment. If you are in a situation where your population of low SES kids exactly equals the population of high SES kids, your TAKS results will be (90% + 45%) divided by 2, which is 67.5%. Obviously in this scenario you have problems. If your low SES kids are a larger population than your high SES kids, they will count more (not equal as in the above example) and will draw your scores even lower. You are in serious trouble in this example, and your trouble grows worse with higher percentages of low SES kids.

So, on common assessments, which are statistically invalid, I compare groups. Low SES to High SES. African American to White. Take your pick depending on your demographics. Give the assessment, forget 60, 70, 80, or any other silly number. They are all arbitrary numbers picked out of the sky. Take the average scores of your high SES kids on your common assessments. Let's say your high SES kids scored an average of 55% correct on your common assessments, but your low SES kids score on 20% on the SAME common assessment. You just measured a 35% achievement gap on your common assessment. BUT WAIT; didn't we have a 45% achievement gap on TAKS? 45%, 35%, those numbers are in the same ballpark!

Now focus on shrinking the achievement gap between your low SES and high SES scores. On the next assessment your high SES kids may average 95%. Great, right? Maybe. Your low SES may score 70% on the same exam. Great news! The low SES kids made 70%! But wait, that is still a 25% achievement gap. That means if your high SES still scores 90% passing rates on TAKS, your low SES kids are only at 65%! You met 70% on your common assessments, but you still go academically unacceptable.

On your next common assessment your high SES average is only 50%, but your low SES average is 45%. Is that good? Probably. The gap between the two is very low. Any achievement gap when comparing a high performing group to a low performing group that is less than 10% is probably pretty good.

I know you think this is hard to do, but it is not. I have seen great teachers actually reverse the achievement gap and have low SES students outperform high SES students. The TAKS results? In a real world, Unacceptable school, I gained 17% low SES performance in science and 10% low SES performance in math in one year, and that was with 1/2 of my math department on growth plans.

The result? That school went from Unacceptable to Recognized (with TPM) in one year.

SC Response

When it comes to common assessments, our points of agreement are in excess of over 90%.

I would have to talk to you to get a better feel about your 70 or 80 point. I’m an 80% guy. But for me, the 80% cut score is simply a way to determine a level of assumed mastery. At 80% mastery, I can gamble that the student is ready to move forward. Below 80% and I need to engage in some re-teaching activities. But remember, I don’t use common assessments for grades; I use them for information to drive planning and instruction. That is an understanding that is not readily apparent to those new to the common assessment game.

You are dead on, with the issue of validity (invalidity). But like you, I’m ok with that. The process of measuring something, even when we know we are measuring it poorly, puts us in a position to measure it better the next time. To not measure is to either give up or rely on luck. If you find either of those to be a viable course of action, then you are not a LYS’er.

I’m also with you on tracking the gap between demographic groups. You use the assessments to determine if your actions and interventions are closing the gap. If you aren’t looking to identify what you will change, then you are not harnessing the power of the common assessment process.

I will point out that if you are actively looking to identify the practices that will close the achievement gap and are in fact implementing those identified practices, that in some cases you will actually increase the size of the gap. The reason for this is that real improvements in instructional quality can raise the performance ceiling faster than it raises the floor. Which is why we focus on improving adult practice instead of circumventing it. You end up with more scars (as the two of us can attest), but the success is much more dramatic and gratifying.

Think. Work. Achieve.

Your turn...

Follow Sean Cain on www.Twitter.com/LYS Nation

Attend the LYS Presentation at the National Conference on Education (2/19/2011)

Attend the LYS Presentation at the TASB Winter Legal Conference

Visit the LYS Booth at the NASSP Conference

Attend the LYS Presentation at the Texas Middle School Association Conference

A Reader Writes... (Game On! Schools - Part 5)

In response to the 12/7/2010 post, “Game On! Schools – Part 3,” a reader writes:

I am all for a little competition, but this sounds just a little aggressive. I would like to know how Game On schools manage their time, and of the entire day what part of their time is most valuable to them or that they manage better than any other time? What is their secret to being so successful?

SC Response

Yes, the Game On! schools and staff are aggressive. They are aggressively chasing down excellence. Excellence defined terms of student and campus performance. Part of this chase does involve managing time. The simple version of this is that the Game On! schools work on the things that matter and quit working on the things that do not matter. That is how you effectively mange time. Your “To Do” list becomes more focused and your “Do Not Do” list becomes increasingly expansive.

At a Game On! school, what is most valuable? Teaching, learning, and measurable performance. What is least valuable? Anything that gets in the way of teaching, learning and measurable performance.

So what is the secret? The Game On! schools operate a structured PLC system that, to use a Good To Great metaphor, constantly rinses the cottage cheese. That system and the insights that it forces staff to develop, drive student performance beyond what can be expected in traditional settings. If you are interested in implementing Game On! on your campus, send me an e-mail.

Think. Work. Achieve.

Your turn...

Follow Sean Cain on www.Twitter.com/LYSNation

Attend the LYS Presentation at the National Conference on Education

Attend the LYS Presentation at the TASB Winter Legal Conference

Visit the LYS Booth at the NASSP Conference

Attend the LYS Presentation at the Texas Middle School Conference

Happy Birthday to the LYS Blog

That’s right LYS Nation, our daily conversation is two years old today. It alternately feels like we just started this yesterday and that we have been doing this forever. I have to admit when I started writing two years ago I had no expectations and no idea where this would take us. My biggest fear was that I would have writer's block after the first two weeks. Fortunately, the blog struck a cord with you (the reader and practitioner) and the LYS Nation was born. Writer’s block has yet to be an issue because the LYS Nation quickly stepped up and took over the topics of discussion. And as I regularly remind you, it is much easier and rewarding to participate in a dialogue than support a never-ending monologue.

I thought it would be fun to re-visit the first post I wrote for the blog, and yes, exactly two years later, I’m dashing this off right before I hit the road to visit another school.

Thank you for another great year and who knows where we’ll go in the upcoming year.

Getting Started: An Introduction (February 2009)

Here I am sitting in another airport terminal at 6:00 in the morning. I wish I could say that this was unusual, but it’s not. I’m now a school road warrior. For the past 5 (now 7) years I have lived on the road, 3, 4 and too often 5 nights a week. Going where schools and principals have needs and problems that they need help with.

There are some perks. Because of the travel points I’m a Hilton Diamond member and a Continental Elite member. This means on a big jet and in a big city, I get upgraded. That happens a couple a times a month, but most of the time I’m on a small plane going to a small town. I also get a lot of free Southwest Airline tickets. Congratulations, you fly a lot, do you want to fly some more?

I’m not complaining. I’m just making the case that I have seen a lot, worked with a lot of principals and schools, and have fixed a lot of problems. All of that to say, that what I’ve seen, what I’ve learned, and what I do may be useful out there.

I have observed that school leaders for the most part live on islands. Islands that have been built by isolation, misinformation, wishful thinking and/or petty jealousies. Hopefully, I can help get some of you off the island, or at least make the island more hospitable.

So here’s what I’m going to do. I’m going to write about school leadership and school operations. The focus will mostly be on improvement and what works. But I’ll also write about the tools that I find useful, books that I have read, conversations that I have had and respond to your comments. Who knows where this will go, but I hope that every once in a while, you’ll find something that is useful to you, your school and/or your staff.

Time to board now, off to another city and another school.

Think. Work. Achieve.

Your turn...

Follow Sean Cain on www.Twitter.com/LYS Nation

Attend the LYS Presentation at the National Conference on Education (February 19, 2011)

Attend the LYS Presentation at the TASB Winter Legal Conference

Visit the LYS Booth at the NASSP Conference

Attend the LYS Presentation at the Texas Middle School Association Conference

A Reader Writes... (Quit Wasting Time - Part 1)

In response to the 12/6/2010 post, “Quit Wasting Time,” a teacher writes:

The assumptions you hold are disturbing and incorrect. I strongly disagree with what you call "a waste of time." Celebrations really matter to children. Spending 30 minutes twice a year for holiday parties is not a waste of time. Fun actually matters to children and should not be summarily dismissed as frivolous. Field trips are not a waste of time. Most of life occurs outside the classroom and the students I teach have few experiences outside their impoverished homes. Going places and experiencing new things outside the classroom is exactly what they need. Stimulation outside a classroom is exactly what they need. The kindergarteners I teach were not allowed to go on a harvest-themed field trip because it wasn't "educational." Yes, it was. They would have benefitted by seeing an actual corn plant and pumpkin patch. They need to know the parts of a plant and how they grow. They need to hear vocabulary like "crops" in an actual rural setting. My second graders didn't know what a pine cone was until I took them to a pine tree and showed them and let them hold a pine cone and look at its parts. These students need field trips more than the upper middle class kids from other schools who all got to go on that field trip. I encourage you to reconsider what you call a waste of time.

SC Response

I won’t even try to argue with you on this one. I will just point out that the most valuable asset that we have is time and I have yet to meet a teacher who has told me that they have too much of it. If fact, I was talking to a large group of educators recently about the challenges that they face, and by a huge margin, the lack of time was their biggest concern. Every second that is not devoted to teaching the content has to be evaluated in terms of performance benefit and cost.

If time is not an issue on your campus, you are fortunate. If time is an issue, search everywhere for it and protect it when you find it. Time hides everywhere and everyone has a valid reason for just stealing “just a few” minutes.

Think. Work. Achieve.

Your turn...

Follow Sean Cain on www.Twitter.com/LYSNation

Attend the LYS Presentation at the National Conference on Education (2/19/2011)

Attend the LYS Presentation at the TASB Winter Legal Conference

Visit the LYS Booth at the NASSP Conference

Attend the LYS Presentation at the Texas Middle School Association Conference

A Reader Submits... Grades

An old school LYS’er submits:

The Assault on Educational Mythology, Continued

OK, now that we have beat up homework, its time to move on to another mainstay of education, grades. Go ahead and brace yourself, this might hurt.

Every bit of research I have seen, both anecdotal and valid, indicates there is NO correlation between the grades we assign and student achievement on standardized tests. Yet we continue to assign grades in schools as if they really mean something. How can this be? Let's look at it.

First, most teacher assigned grades come from tests, which the teacher creates. These tests are created by the teacher to assess what the teacher taught, of course. Whether or not the teacher actually taught something to state standards is irrelevant in this case. We all think we are great test writers, but I have yet to meet a teacher who is an expert test writer, as test writing is actually a profession in its own right. Same goes for curriculum by the way, but that is another story. Second, the remainder of the grades generally comes from teacher created assignments, such as homework.

Having established some main sources of grades, lets ask ourselves what a grade means. Ideally a grade means that the student mastered the material presented. Hopefully the material presented was aligned to state standards, so the grade should indicate the student mastered state standards. But we have already established there is NO correlation between student grades and state assessments, assessments that are certainly aligned to state standards.

So it appears in most cases grades have little meaning to learning that is aligned to state standards. Let's assume you are an exception and ALL of your assignments are meaningful, well thought out, and are true indicators of a student's mastery of state standards. If that is the case, those are some important assignments, VERY important assignments. In fact they are so important, the student must do them in order to be successful in your class and on state assessments.

So a student doesn't do this very important assignment, you wait for the appropriate amount of time to pass as indicated by district policy, and then you assign a ZERO for this very important assignment. Seriously? By assigning a ZERO for an assignment that is truly important and critical to student success, you have just sent the message that the assignment truly had no value at all. If your assignment were truly that important, you would take every step humanly possible, including parent contact, home visits, and office referrals, to get the assignment completed.

Let's also talk about "daily" grades in general. What if a child fails every daily assignment for a weeks, yet near the end of the unit actually "gets it", does well on only two of your "daily" grades, yet passes an assessment with an 80 that is truly aligned to the district curriculum, ergo the student learned the material. In this case, congratulations, you did your job as a teacher. However, all of those failing daily grades will likely bring the grade down to a low C, or maybe even an F. In this case, what did you actually measure with your daily grades? In this case, and in most cases, daily grades measure how a student learns an objective and how long it takes the student to learn an objective. In the above example, if you assign a grade any less than the 80 for the grading period, shame on you. Should we really grade students on how they learn and how long it takes them to learn? Of course not!

So there we have it. Either your assignment is like most and is totally unreliable for indicating success on state assessments, so assigning any grade at all for the assignment is questionable. In this case giving failing grades, particularly a ZERO, seems particularly silly.

Or your assignment is good but the grades collected are actually measuring how a student learns and how fast the student learns.

Or your assignment is unlike most and is very aligned to state standards and is critical for the student to succeed not only in your class, but also on state assessments. How could you EVER decide to record a zero, or for that matter a failing grade, for such an important assignment?

So either way you take it, your grades are probably meaningless. Stop focusing on grades and start focusing on the needs of learners and their mastery of the district curriculum.

SC Response

Anyone who does not believe that grades are subjective is simply fooling oneself. I always chuckle when a read someone defending the sanctity of their assigned grade. All a grade does is give an indication of how well the student played the teacher designed game and/or how compliant the student was in a particular class (I myself was more guilty of this than I like to admit). Like the writer, there is no need to bore you with a summary of the literature that supports this, you either intuitively understand this or you don’t. If you understand this, you search for and use other metrics to determine student success. If you don’t, you continue to tilt at windmills.

As you briefly touched on, as teachers we have a difficult time creating valid tests. Now before the flurry of contrary comments come flying in, let me explain why this is the case.

1. As teachers, we create tests based on what we have taught, not what we were supposed to teach. But to do otherwise, would unfairly punish our students for the pace of instruction. This is a no win situation for the teacher. Compromise your students or compromise your test.

2. As teachers, we provide weak (easy to identify) distracter answer choices. This compromises the validity of the test.

3. As teachers, we test primarily at lower levels of rigor.

As you point out, creating a valid test is a profession and requires technical expertise. This does not mean that teachers can’t eventually become expert test writers, but I argue that it is a waste of teacher time.

If at the end of the day, the teacher has one hour of preparation time. I would much rather the teacher spend that hour working on how to better deliver instruction instead to trying to create a better test. Trust me, the professional test writer isn’t spending time trying to figure out how to teach better.

Creating a test for just my class is one of the traditional activities that teachers need to let go of, either voluntarily or by mandate. Note: This is not an argument against common assessments, it is an argument for common assessments.

I also agree with your observation that most grades are simply an indication of how quickly the student grasped the material. And really, how fast one grasps the material is not only of little relevance, it unfairly penalizes the student with fewer out of school resources. It might be a coaching philosophy, but during practice time all I want is effort and improvement. I’ll assess (and be assessed) based on game results. Apply this to the classroom and it means that during the week I want effort and improvement in mastering the material, but it is actually performance on the common assessment, cumulative final, or state assessment that really gauges the success of the student and my effectiveness as a teacher.

I’m not advocating getting rid of grades or giving away grades. I just realize that as a source of valid information, relying on reported grades is as useful as getting all of your news from just one source. There is always a bias.

Think. Work. Achieve.

Your turn...

Follow Sean Cain at www.Twitter.com/LYSNation

Attend the LYS Presentation at the National Conference on Education

Attend the LYS Presentation at the TASB Winter Legal Conference

Visit the LYS Booth at the NASSP Conference

Attend the LYS Presentation at the Texas Middle School Association Conference