Brian Davison mentioned the other day that it can be useful to follow the Pass Advanced and Pass Proficient numbers that are elements of the total pass rates.
The Board of Education has set “cut scores” for the various SOL tests. For instance, on the 5th grade math test, a student needs 31 of 50 items to score Pass Proficient and 45 of 50 for Pass Advanced. The VDOE database reports the Advanced and Proficient pass rates down to the school and grade level.
For my first venture into this thicket, I’ve pulled the reading and math subject area data by grade for Richmond and the state.
Let’s start with reading. Here are the 5th grade data:
To untangle this, start with the red data points, which are Pass Advanced rates for Richmond (yellow lines) and the state (blue lines).
Richmond fifth graders were performing within about 10% of the state average before the new tests in 2013, actually much closer in 2012. The new tests in 2013 halved the Pass Advanced rate for the state and cut Richmond by two thirds.
The green points are the Pass Proficient data, again with yellow lines for Richmond and blue for the state. Before 2013, Richmond was partially compensating for its lower Pass Advanced rate with a higher than state average Pass Proficient rate. The new tests in ‘13 dropped the Richmond Proficient rate by almost a third but left the state rate essentially unchanged.
So, at the state level, we see the new test moving about half of the Pass Advanced students to lower tranches (presumably mostly to Pass Proficient) and moving about as many pass proficient students to failing as were added from the ranks of the former Pass Advanced. In Richmond, however, the Pass Proficient rate dropped by almost 20%; that rate recovered a bit two years later but it remains below the state average.
The curves with no points are the total pass rates, blue for the state and yellow for Richmond.
Why did our elementary schools (and middle and high schools, see below) get disproportionately clobbered by the new, tougher, tests? Almost certainly because Richmond’s then-Superintendent did not align the curricula to the new tests.
This handed our current Superintendent a blank check: Fix the curricula and get a bounce. (Bedden started here in January 2014, so look for the Bedden Bounce in the ‘14-15 numbers.) In these data, the Pass Proficient rate shows some bounce but the Pass Advanced rate does not.
Bounce or no, the new tests lowered Richmond’s overall pass rate, compared to the State. Richmond’s fifth graders had been flirting with the state total pass rate until 2013; now, even after the Bedden Bounce, they are fourteen points behind.
Next the 8th grade reading data:
The new tests whacked the Pass Advanced rates both in Richmond and statewide. The Pass Proficient rate for the state jumped under the new tests (looks like a chunk of the 40% decrease of the Advanced population wound up in Proficient) but in Richmond that rate fell. Again, the Pass Proficient rate in Richmond enjoyed a bounce under the new Superintendent, but left us ten points below the 2012 level. The Pass Advanced numbers rose slightly in ‘15.
As to the totals, the new test clobbered Richmond’s (already dismal) rate vs. the state, leaving our eighth graders thirty points down.
The End of Course numbers show a larger 2013 hit to the Pass Advanced rate, both here and statewide. The state Proficient rate jumped; the Richmond rate showed a smaller rise with no Bedden bump:
Our EOC average in 2016 was fourteen points below the state average.
Turning to math: The new tests came a year earlier, 2012. Those tests degraded a close-to-average fifth grade performance.
Again, the Pass Proficient rate enjoyed a bounce in 2015. The 2014 increases in the both rates came after Bedden had been on the job for only six months, so the credit will have to go elsewhere. As to the totals, after the Bedden Bump we still are nine points down.
Our awful middle schools blew the new math tests, big time.
After the Bedden Bump (Proficient only), our total pass rate still was thirty points below the state average.
The Pass Proficient rate in the high schools partially recovered from the new tests in 2014 and ‘15; the Pass Advanced rate, not so much.
In 2016 our total pass rate remained sixteen points below the state average.
A look at the averages by subject area emphasizes the magnitude of Richmond’s failure here, particularly in the Pass Proficient rates:
We were twenty points below the state average in reading in 2016, 23 points in math.
For the global view, here are the five-subject averages:
The 2016 Richmond average is 22 points below the state average.
Note: Jim Weigand points out that 2015 also was the first year that retakes were allowed for the elementary and middle school grades. VDOE does not release retake data; they did admit: “The 2014-2015 school year was the first during which students in grades 3-8 were allowed to retake SOL tests in reading, mathematics, science and history. On average, the performance of students on expedited retakes increased pass rates by about four points on each test.”
In light of that, we probably could discount the statewide average increases for 2015 by about 4% and we could wonder how much of the Richmond increases was Bedden Bounce and how much was retakes.
Here are the raw numbers:
Bottom line: The modest gains from aligning the English and math curricula to the new tests mostly failed to repair the damage those tests did to Richmond’s’ pass rates.
That left RPS mired in failure: We have the lowest pass rates in the state in reading and the five-subject average; we have the second lowest in math, writing, history & social science, and science. That tells us something crucial about our Superintendent and about the School Board that hired him.