A Reader Writes... (Common Assessments for Students with Special Needs)

In response to the 12/9/2010 post, “Common Assessments for Students with Special Needs,” a LYS Principal writes:

OK, the confusion and misapplication is amazing.

First, everything the special education director said is 100% accurate. I had this same discussion with my special education director and made the decision to allow accommodations/modifications to special education student scores but to NOT count them in my common assessment data analysis. I did this because we count the common assessments as a grade. I have done it both ways. That is, I have counted common assessments as grades, and I have not. I don't have an opinion on which way is best, but if you count them as grades follow the special education director's advice.

Second, I once wrote on this blog to not fixate on 70 when determining passing rates. In response I have had people tell me, "We don't use 70%, we use 80%". I have news for you - that is not better. It is likely that is worse.

I possess a gift that shines above that of most other people. I understand math at a very deep level. I was a physicist and research scientist for a major university before entering public education. With that fact behind us, let's get this one out of the way - your assessment is likely "invalid" in the statistical sense. That doesn't mean it is useless. "Invalid" used in this context is a technical term, not a judgment.

The next thing to get out of the way is that once you finally decide to accept the fact that your assessments are invalid, you can then move on to the fact that it is virtually impossible to compare performance on one invalid assessment to another invalid assessment.

Many at this point, then ask, “Why bother?” The answer is simple. Use your common assessments to compare the performance of one group against a different group on the SAME exam. Even if the exam is invalid (yours are, accept it), it is still reasonable to look at the performance of two different groups on the SAME assessment.

Example: Your high SES kids are scoring 90% or better on all levels of TAKS. Your low SES kids are performing 45% on average on TAKS. You have a 45% achievement gap on a VALID assessment. If you are in a situation where your population of low SES kids exactly equals the population of high SES kids, your TAKS results will be (90% + 45%) divided by 2, which is 67.5%. Obviously in this scenario you have problems. If your low SES kids are a larger population than your high SES kids, they will count more (not equal as in the above example) and will draw your scores even lower. You are in serious trouble in this example, and your trouble grows worse with higher percentages of low SES kids.

So, on common assessments, which are statistically invalid, I compare groups. Low SES to High SES. African American to White. Take your pick depending on your demographics. Give the assessment, forget 60, 70, 80, or any other silly number. They are all arbitrary numbers picked out of the sky. Take the average scores of your high SES kids on your common assessments. Let's say your high SES kids scored an average of 55% correct on your common assessments, but your low SES kids score on 20% on the SAME common assessment. You just measured a 35% achievement gap on your common assessment. BUT WAIT; didn't we have a 45% achievement gap on TAKS? 45%, 35%, those numbers are in the same ballpark!

Now focus on shrinking the achievement gap between your low SES and high SES scores. On the next assessment your high SES kids may average 95%. Great, right? Maybe. Your low SES may score 70% on the same exam. Great news! The low SES kids made 70%! But wait, that is still a 25% achievement gap. That means if your high SES still scores 90% passing rates on TAKS, your low SES kids are only at 65%! You met 70% on your common assessments, but you still go academically unacceptable.

On your next common assessment your high SES average is only 50%, but your low SES average is 45%. Is that good? Probably. The gap between the two is very low. Any achievement gap when comparing a high performing group to a low performing group that is less than 10% is probably pretty good.

I know you think this is hard to do, but it is not. I have seen great teachers actually reverse the achievement gap and have low SES students outperform high SES students. The TAKS results? In a real world, Unacceptable school, I gained 17% low SES performance in science and 10% low SES performance in math in one year, and that was with 1/2 of my math department on growth plans.

The result? That school went from Unacceptable to Recognized (with TPM) in one year.

SC Response

When it comes to common assessments, our points of agreement are in excess of over 90%.

I would have to talk to you to get a better feel about your 70 or 80 point. I’m an 80% guy. But for me, the 80% cut score is simply a way to determine a level of assumed mastery. At 80% mastery, I can gamble that the student is ready to move forward. Below 80% and I need to engage in some re-teaching activities. But remember, I don’t use common assessments for grades; I use them for information to drive planning and instruction. That is an understanding that is not readily apparent to those new to the common assessment game.

You are dead on, with the issue of validity (invalidity). But like you, I’m ok with that. The process of measuring something, even when we know we are measuring it poorly, puts us in a position to measure it better the next time. To not measure is to either give up or rely on luck. If you find either of those to be a viable course of action, then you are not a LYS’er.

I’m also with you on tracking the gap between demographic groups. You use the assessments to determine if your actions and interventions are closing the gap. If you aren’t looking to identify what you will change, then you are not harnessing the power of the common assessment process.

I will point out that if you are actively looking to identify the practices that will close the achievement gap and are in fact implementing those identified practices, that in some cases you will actually increase the size of the gap. The reason for this is that real improvements in instructional quality can raise the performance ceiling faster than it raises the floor. Which is why we focus on improving adult practice instead of circumventing it. You end up with more scars (as the two of us can attest), but the success is much more dramatic and gratifying.

Think. Work. Achieve.

Your turn...

Follow Sean Cain on www.Twitter.com/LYS Nation

Attend the LYS Presentation at the National Conference on Education (2/19/2011)

Attend the LYS Presentation at the TASB Winter Legal Conference

Visit the LYS Booth at the NASSP Conference

Attend the LYS Presentation at the Texas Middle School Association Conference