Student test scores came out a few weeks ago and once again districts, media, and pundits (us included), made lists of schools and districts to watch. It seems too often these days, education reformers get ahead of themselves declaring victory (with a year of growth data) or nothing at all before we know whether most students have the basics of reading. Every year there appear to be a new list of schools that have “made it” or are “worth watching” even though they have had only one year of positive results.
At the other end of the spectrum, many schools, in spite of only very modest improvement, came off the state watch list while others fell onto the list again by equally narrow margins. Not surprisingly, schools and districts that have performance near the state’s cut points tend to bounce up and down in rating over time, dependent not only on their performance but how others are performing relative to the school or district in question.
All of this reminds me a bit of high school when certain kids stood out positively, sometimes for good reasons and sometimes just because they were “popular” or loud in some way. Those of us that have returned to any high school reunions after twenty years or so know the accuracy of any predictions back in our senior year are not only limited but often wildly inaccurate.
I worry that the latest 2017 “it” schools as promoted by themselves or their districts may be more like the collective high school predictions about post-high school life than some definitive objective measure of the schools that will deliver consistent results for kids. There is no question that some of the “it” schools have had some remarkable achievement scores and are likely to carry on in the coming years while others are just “popular” for any number of reasons. Regardless of which category these “it” schools fall into, nearly all need more time before they should be considered effective schools.
The entire online schools’ movement should be a powerful lesson for the education reform community’s over exuberance on an innovation that not only did not deliver what was promised but has led to schools far worse than some of the state’s lowest performing brick and mortar schools.
You can return to the Denver Post or Chalkbeat articles in just about any year just after the release of test score data to find the “it” schools of a particular year getting pages of press in their first year only to be soundly forgotten about when the new “it” schools come online and some of the previous years’ schools fail to live up to expectations.
It is not uncommon to find some of the state’s current worst schools to have been talked about as remarkable turnarounds sometime in the last decade because of an upturn in performance, a new program or leader.
Can’t help but think of Billy Joel’s Scenes from An Italian Restaurant:
“Brenda and Eddie were the popular steadies. And the king and the queen of the prom. Riding around with the car top down and the radio on. Nobody looked any finer. Or was more of a hit at the Parkway Diner. We never knew we could want more than that out of life. Surely Brenda and Eddie would always know how to survive.”
We all know the story too well, poor Brenda and Eddie “got a divorce as a matter of course. And they parted the closest of friends. Then the king and the queen went back to the green. But you can never go back there again.”
Sounds like school turnaround all over again where we are quick to pick superficial winners, throw lots of accolades and before you know it, they’re back to square one, forgotten by the media and reformers.
What is happening?
- School supporters are quick to declare success sometimes amplified by affinity groups, boards, and investors.
- The media often has a short memory and does not have the capacity to go beyond a simplistic look on one stream of data for one year. Our own attention as consumers of this information is short, rarely do we review trends in the context of timelines with interventions like a new leader or program.
- School districts are often desperate for good news and will look for any success regardless of whether they know the cause or if they can sustain it beyond a year or two.
- Last, there may be a bit of a school evaluation paradox. While it is becoming easier to predict if a school will fail early with indicators from a school culture and interim achievement data, it takes time to know whether a school will succeed because of the need to have several years of achievement data with trends.
We are fortunate to now have more data on schools than ever before (even though some of this data recently is being unfairly suppressed by the state). We can look at school culture, attendance, interim assessments etc. and first year achievement data to provide insights into whether the school is on track to long-term success. It is difficult, however to know whether a school is really working until we have several years of achievement data showing that students are reaching standards. Confusing early indicators with demonstrated effectiveness leads to what I would describe as a “false positive.” You can get things right in the first year or two, which is necessary but does not always translate to sustainable success.
While it’s understandable why school districts and others may want to claim a particular school works given pressure to perform (and pressure on balance is good in that it has led to some great new effective school models), we all have to be far more careful in not reinforcing false claims before having more substantial evidence that a school is working.
We also need to have far more rigorous and precise processes for knowing when to monitor, double down or shut down a school or strategy for improvement. We need a smarter long-term view on improving or developing great schools as we also must have more information on the quality of schools for policy makers and families. There is often a disconnect between when the information is available, what is required for school success and when policy makers need to decide what to do with a new or struggling school.
There is no question that we need to support innovation and new school models given the fact that currently, thousands of students in Colorado have little shot at college, a career, or a living wage in today’s economy (over 38,000 Colorado third grade students are not reading at level). Our desire for improvement requires us to be more vigorous about declaring success while we also doing more to support innovation which will require more discipline in understanding why something is working (the scientific process).
Creating great schools is far more complicated than rocket science. It’s far easier to build a rocket than a great school that serves all students well. The good news is that smart committed leaders are creating effective schools and also improving them throughout the country. It’s a small but growing number. Doing this work well requires a level of work, creativity, design thinking, management and love that is a challenge to replicate.
It’s time we move beyond the simple minded quick fix solutions for enormously challenging problems. Let’s make sure that we really understand what Brenda and Eddie are doing that will lead to success instead of banking on their popularity to succeed. We need to make sure our evaluation and reporting on these schools becomes more accurate and nuanced so that we do not continue to make the same mistakes over and over again.