A few weeks ago, education headlines all over Colorado read “PARCC Results In.” Summarized, the articles asserted that Colorado elementary schools had shown slight improvement from 2015 to 2016, but the results were vague to say the least. And it’s true, while the Colorado State Board of Education released the second year of PARCC state test results, it wasn’t until a few weeks later that we had any information about school or district level results. And even those results were far less complete than ever before, sparking confusion and frustration across the state. Indeed, with the last release of school and district level results, over 4,000 lines of data were masked. Could you imagine the Centers for Disease Control suppressing the number of ZIKA cases in Miami Beach because of HIPAA? It is as if the state has decided that nearly 30% of the ZIKA cases or cures cannot be reported.
Unfortunately, we will just have to wait for more useful information like growth percentiles and information about different groups of students in schools and districts, while the state works out a series of issues related to the adoption of the new test. We were told similar stories last year and did not receive much of the data until over a year after the test was given. Who knows when teachers or parents will get the results from 2016, which will probably not be very helpful as the new school year has already started. It also remains unclear whether PARCC will ever provide the level of detailed data to the public that we used to have with CSAP to pinpoint which programs, schools or districts are working well for kids and which ones are not.
PARCC’s Promises: It Was Not Supposed to be This Way
PARCC was assumed to be the “Ferrari” of testing, as Colorado State Senator Michael Johnston aptly put it to the State Board of Education five years ago. A number of national experts sold PARCC and the competing Smarter Balanced Assessment Consortium as a quantum leap in the quality of state tests.
We were going to get the test results back sooner (before the end of the school year) and have a test that not only assessed the same sorts of knowledge as CSAP but also measured higher order problem solving. We would be able to compare Colorado results to dozens of other states. The data would be easier to access than CSAP had been. We thought PARCC would transform the ways we help Colorado schools improve.
We were Love-Drunk on PARCC…and Now We’re Hungover
The need for a new assessment coincided with the change in what we were asking kids to do. CSAP–Colorado’s first statewide assessment for all students that began in 1997–was not very effective at measuring higher order thinking skills, did not assess the new Colorado Academic Standards of 2009, nor provided useful information to drive learning at the individual level; however, it did provide information to inform school and district level change. CSAP marked progresses, showcased successes, and assisted in goal-setting for the future.
Colorado did need a new assessment–one that was more rigorous and that provided results from the test in a more timely manner. That was where PARCC came in.
We are now fully entrenched in the implementation phase. And boy, does it feel like the trenches. We were promised more timely results. We were promised more useful information for educators. We were promised more accessible and actionable information for parents.
Instead there is far less transparency on PARCC data then there was with CSAP. For CSAP the state built a powerful database that enabled anyone to easily find achievement levels, mean scale scores, growth over time by year, by any designated student population be it race/ethnicity, gender, free/reduced price lunch eligibility, or educational need in any school or district setting.
Now, with PARCC, we started the new school year without school or district-level results anywhere near CSAP release-caliber, let alone the improvement we were promised. Teachers and school leaders entered their classrooms without their students’ results, having to rely on other sources for information PARCC could provide. Many parents and families across the state (myself included) never got their child’s results from PARCC 2015.
Without clear, accessible school and district-level disaggregated data we don’t know how many low-income African-American students at a particular school are reaching grade level expectations. Or how these students are doing relative to other black, white, or Latino students at other schools. It will also be difficult to understand the impact of other education legislation like the Read Act or any other proposals to improve public education. Anyone that supports a particular change in public education, as well as those who believe the system works except for a lack of funding should be equally concerned about not having real time data on student learning from standardized tests.
I do not know whether to blame PARCC, CDE or Pearson but the result is still the same: we will not know which schools are under or over-performing without doing some fairly complicated analysis. There are complex methodological challenges and data reporting changes that have obscured what we can ask of the data. We have basically gone back a decade or so when we had little real-time valuable information on student learning. We are flying blind all over again, not sure what is or is not working.
You Get What You Pay For: Expensive Tests Earn their Stripes when the Data is Prompt and Comprehensive
While there have been many debates about the costs of one particular test or even the cost of switching to a new test, it is important to remember that regardless of the test, Colorado spends a significant amount of money on testing every year. Here is a back of the envelope estimation of just what that cost is.
Let’s say a typical annual standardized achievement assessment takes about four hours per subject per year. It’s pretty hard to have meaningful instruction on testing days, so let’s estimate an average of three days per student for test administration. That comes to $99,152,605 spent on test administration.* Plus the cost of the PARCC test itself- $24,362,560 (most tests cost between $20-50 a student).
While the grand total of $123,515,165 for the year is quite a chunk of change, I am obliged to mentioned that it still represents less than 1.5% of the $8.8 billion spent on public education in Colorado. To be clear, regardless of which test is administered, Colorado will be spending at least $124 million each year. Wouldn’t it be nice if we got $124 million back in value for these tests?
Hair of the Dog: Remembering the Value of High-Quality Assessment Data
The importance of high standards and aligned assessments–well as the information gathered–has seemingly been lost. It was supposed to be about useful information for parents, students and teachers. And unfortunately it’s not unusual to see a chasm between legislative intent and what happens on the ground.
Here is why I still care about PARCC and you should too.
Full disclosure, my worst experience in high school was taking a standardized test, the SAT. I barely broke 400 out of an 800 scale for reading. My high school average score was 650. I did not read or write much until my last year of elementary school. None of my high school teachers knew anything about my dyslexia or my fears in facing them. My teachers were never told and never asked, and I was a good student that was able to mask the dyslexia. Fortunately for me, I came from a privileged family with an expectation of college so I found myself in a good college in spite my verbal test scores. Kids are so much more than a test but a test can provide valuable diagnostic information when used in the context of other information.
There was a day (twenty years ago in Colorado) when it was nearly impossible to have an objective third party measure your child’s ability to read, as well as your neighborhood school’s ability to teach your child to write. It is a vital combination of personalized, teacher-based feedback and objective test scores that allow families to make thoughtful decisions about where their child is most likely to succeed.
While standardized tests like PARCC, CSAP, ACT or SAT should never be used to showcase the entire picture of student achievement, it is an essential diagnostic tool for understanding a band of academic skills. When used to look across a group of students or a school test scores can provide invaluable information about what the school is teaching or not. Effective use of standardized tests is a necessary component of creating quality public schools.
Colorado’s decision to join PARCC was part of a unique–and perhaps the most visionary–time in state education politics. A coalition of leadership from the right and left passed a flurry of education policy changes that covered high expectations for all students, better measurement of student’s competencies, new legal structures to give schools and districts more flexibility, and teacher evaluation. The last time we saw so many fundamental changes in education policy changes at one time on so many fronts was probably in the 1960’s. But this vision and these ideas fall flat if we can’t implement them correctly.
Time to Get Implementation Right
As I reflect back on all of the well intentioned work to improve Colorado’s schools five years ago, I see that many of us, myself included, were lulled into believing the hype about PARCC. Indeed, education policy advocates, particularly those of us in the “education reform camp,” (even if well-intentioned) love shiny objects (think iPads for All in L.A.) but have a hard time with follow-through. Implementation is hard and boring; it is more fun to pass a new law and have others worry about the details. If we’ve learned anything from the implementation of PARCC, it’s that the devil is in the details.
Five years ago, we did not adequately reflect on how to turn policy into effective practice. We are now beginning to wake up from the hype and now feeling the hangover of PARCC implementation. So as we start the new school year, it’s time to take a deep breath of fresh reality and either thoughtfully rework the PARCC system so it delivers information to families and educators or move onto an assessment that is better suited to meet all of our needs. We can’t afford to spend another $124 million dollars on a test that does not have the infrastructure in place to support innovation in real-time. Colorado educators, families, and students are (still) waiting and deserve better.
*($9,767 per year/student X 609,604 Grade 3-9 Colorado students)/average of 180 days of instruction per year= $54 cost per instructional day) http://www2.census.gov/govs/school/13f33pub.pdf