Lies, Damn Lies, and High School Pass Rates

As I discussed at some length here and here, VDOE does not report SOL scores for Maggie Walker.  For reasons that have nothing to do with the truth, they report the scores of the MLW students at the high schools in those students’ home districts (Note: Old document; today Pearson surely reports the scores directly), albeit the MLW students are entirely educated at MLW, not at the schools where the scores get reported.

Fortunately, the Board of “Education” has not been able to “adjust” the rules of algebra.  So we can calculate estimate the score boosts the Richmond high schools enjoy courtesy of this mendacity.  All it takes is the number of students at the high school, the reported pass rate, the number of those students who actually attend MLW, and the average pass rate at MLW.

The first three numbers are available in the “school quality profiles” on the VDOE Web site (but I got the unrounded pass rates from the very nice VDOE database).  We can infer the MLW pass rates: The two selective Richmond high schools (Community and Open) run mostly at and otherwise near 100% and MLW is still more selective.  We’ll use a 100% SOL pass rate at MLW.

With that, the score boosts pop right out.  To start, the reading tests:

image

Isn’t that nice!  The TJ pass rate jumps from an actual 61.3% to the reported 68.9% courtesy of 133 MLW students who DO NOT attend TJ.  The other score boosts are less dramatic but all in the helpful direction.

image

While we’re at this, let’s look at the other subject areas.

image

image

image

image

Do you suppose the feds know that the Board of “Education” is lying to them about these pass rates?

If you understood that the Accreditation numbers were rigged, do you now understand that, as to the high schools, the manipulation is still worse than you thought?

Your tax dollars at (mendacious) “work.”

Examining Some Assumptions

A thoughtful reader writes:

Following the references in the Ryan book [5 miles Away, Worlds Apart” (2010, Oxford)] I discovered an education book I hadn’t known about: Richard Rothstein’s “Class and Schools” (2004, Economic Policy Institute/Columbia Teacher’s College). The subtitle is “Using Social Economic and Educational Reform to Close the Black-White Achievement Gap.”

Rothstein starts with the 1968 Coleman Report and works forward
reviewing study after study trying to figure out what works –
according to the data. Its got the most data I’ve seen on class
effects in education. Its rather slim: 150 pp, with 50 pp of notes.
The world could use an expanded book like this.


Rothstein attributes most of the achievement gap to inequality (and
claims that we’re stuck with this gap unless we fix inequality). While
we can’t entirely fix things his prescriptions for doing the best we
can are to ameliorate the effects of inequality to the best of our
ability. These include things like after-school programs and summer
programs like wealthier kids have access to – where learning occurs,
important stuff, not always of the SOL-score improving type.

Both Ryan and Rothstein make clear how state’s lower the SOL
test-score bar to improve pass rates and hide how poorly they are
serving at-risk populations. While TJ, for instance, may look like its
SOL pass rates aren’t “too far” behind Freeman’s, Freeman’s kids had
exponentially better “advanced” scores, and are learning lots of new
stuff, while TJ’s kids are focused exclusively of hitting low-bar test
cut offs.

Rothstein addresses hot-button issues like IQ in a graceful and
pragmatic way, and doesn’t flinch from addressing cultural educational
practices. It will at least make you think.

The review of Rothstein by Kenneth Strike in the American Journal of Education (sorry I can’t link to the full version; the copyright holder is more interested in revenue than broad availability) starts out as a further attack on standards.  I characterize that school of thought as “It’s too embarrassing to measure outputs; let’s focus on inputs.”  But then he says:

Rothstein’s observations on these matters seem to me to be correct, but they suggest a line of argument that is not fully carried out. We do not, for instance, know whether these problems with standards-based reform are fatal flaws or fixable glitches. We do not know whether we should abandon standards-based school reform or merely hold reasonable aspirations for what it can achieve.

Interesting stuff.

Money Can’t Buy You Graduates

Here are the 2017 4-year cohort division federal graduation rates plotted vs. the 2016 division disbursements per student.

image

The disbursements do not include facility costs, debt service, or contingency reserve payments.  We’re stuck with 2016 numbers because VDOE won’t post the ‘17 data until sometime next spring.

The slope of the fitted line, –0.2% per thousand dollars, suggests that increasing division expenditures are associated with decreasing graduation rates.  The 0.6% R-squared, however, tells us that the two variables are essentially uncorrelated.

Richmond is the gold square on the graph.  The red circles are, from the top, the peer cities Newport News, Hampton, and Norfolk.  Charles City is green; Lynchburg is aqua.

For a look at the other end of the academic process, here are the dropout rates plotted vs. disbursements.

image

The slope again suggests a counter-intuitive result: dropout rate increasing by 0.3% per thousand dollar expenditure increase.  But, also again, the R-squared tells us that the correlation of the two variables is negligible. 

The color codes are the same and the previous graph except the peer cities are reversed: from the top, Norfolk, Hampton, and Newport News.

Finally, while all those data are together in a single Excel file, let’s plot the dropout rate vs. the graduation rate.

image

At last, a correlation.  And one that makes sense: The divisions that graduate more of their students lose fewer as dropouts.  BUT, of course, the correlation does not imply that the higher graduation rates cause the lower dropout rates, or vice versa.

The color codes are the same as above.  The peers, from the left, are Norfolk, Hampton, and Newport News.

Lies, Damn Lies, and Graduation Rates

To follow up on this morning’s post, VDOE’s “on-time” graduation rate counts the

  • Modified Standard diploma (students with disabilities who “are unlikely to meet the credit requirements for a Standard Diploma”), the
  • Special diploma (term not defined on the VDOE Web site but appears to apply to the Applied Studies diploma, available to certain students having a disability), and the
  • General Achievement diploma (none reported this year)

in addition to the standard and advanced diplomas. 

This gives a nice boost to the official “graduation rate,” especially for those divisions willing to misclassify students as handicapped in order to boost their SOL pass rates.  On the 2017 4-year cohort data, the “on-time” fiction boosted the statewide rate by 2.8% and the Richmond rate by 6.7% compared to the federal (advanced plus standard diploma) rate.

image

The boosts this year look to have mostly come from the special diplomas.

image

Actually, it’s worse than that.  Beginning with students entering the ninth grade in 2013-14 (i.e., this year’s 4-year cohort), there was supposed to be no modified standard diploma.  Instead, “Credit accommodations allow students with disabilities who previously would have pursued a Modified Standard Diploma to earn a Standard Diploma.” 

This change has three benefits for the education establishment:

  • The modified standard diploma students who formerly would not count toward the federal graduation rate now count,
  • The divisions have a new avenue – “credit accommodations” – for boosting the rate, and
  • The process is hidden from the public.

This explains the low Modified Standard rates this year.  Last year, those rates were 1.4% for the state and 5.6% for Richmond; this year, 0.1% and 0.5%. 

Looks like this year they successfully concealed about a 5% boost in the Richmond rate.

(The Modified Standard rate should be zero this year except that they get to fudge the cohort for students with disabilities.)

The only question here is whey they did not similarly transform the “special” diplomas into standard diplomas so they could conceal the whole, sordid process.

Added Note:  If we were to figure that Richmond’s federal graduation rate was boosted this year by about 5% (and the state by about 1.3%) because of the transformation of modified standard diploma graduates into standard diploma graduates, Richmond’s 69.9% rate this year

image

would look more like a 65.  If that were the case, Richmond’s recent swoon

image

would look more like a slump (and the state wouldn’t look so hot either).

image

2017 Cohort Data for Richmond Schools

Here are the 2017 4-year cohort dropout rates for those Richmond schools with a graduating class (except for Ric. Career Ed., which had too few students to report).

image

“Mercy,” you say.  “Where did that 18% overall rate come from?”

Well, in addition to the 84 dropouts at Huguenot, there were 70 (in a cohort of 116) at Richmond Alternative.

(As to Alternative, can you spell “dumping ground for troublesome kids”?  Do you remember when it was improving nicely until RPS took it over and embarked on the path to awfulness?)

Here also are the diploma rates.

image

We’re No. 1 (In Dropouts)

The 2017 4-Year Cohort graduation and dropout rates are up on the VDOE Web site.

We have the highest dropout rate in the State.

image

In terms of a graph, here is the Richmond dropout rate, along with those of the peer cities Hampton, Newport News, and Norfolk, as well as Charles City and Lynchburg (where I sometimes have a reader) and the state average.

image

VDOE did a press release to brag about the “on-time” graduation rate.  As with the Accreditation process, they have jiggered the numbers.  The federales only count real (i.e., advanced and standard) diplomas (see the list here) and the data allow us the calculate that uninflated number.

You might think that our awful dropout rate would serve to improve the graduation rate. (Oops!  That was written early in the morning before my old brain turned on.  Dropouts cannot improve the graduation rate; they can – and probably do – improve the SOL scores insofar as the dropouts probably were not great scholars.)  Be that as it may, in terms of the federal rate Richmond was last among the Virginia divisions.

image

Here are the Richmond diploma rates along with those of the divisions in the previous graph.

image

Not only is our total graduation rate in the cellar, we achieved that rate with a preponderance of standard diplomas, in contrast to the average.

The statewide graduation rate has been rising in recent years, driven by increases in the rate of advanced diplomas.  Until the last four years, Richmond has seen increases in both rates.

image

If we fit least squares lines to the totals, we see the state rate increasing at 1.1% per year while the Richmond increase has been 1.6%.

image

If these rates were to continue, the state average would hit 100% in 2028; Richmond would reach the current state average in that same year.  On the other hand, if the Richmond rate were to continue to change at the same rate as the last four years, it would never again rise above 70%.

The 416 Richmond students that did not receive standard or advanced diplomas were 30.1% of the Richmond cohort.  Of those 416, 249 (60%)were dropouts.

Middle School Miasma?

In Richmond, the SOL pass rates drop precipitously between the fifth and sixth grades.

image

I was chatting with a Chesterfield middle school teacher the other day.  He told me that, in his experience, it is a matter of family environment: The kids who go home to a milieu run by adults manage the transition to middle school (and puberty and the social upheavals of that period) much better than those who go home to a situation dominated by a peer group.

To the extent that economic disadvantage (“ED”), or the lack of it, correlates with those family environments, we have some data on that. 

Here, to start, are the 2017 Richmond and Virginia SOL pass rate changes from fifth to sixth grades.

image image

Hmmm.  It looks like there may be something there in the statewide averages on the reading tests but not the math.  In contrast, both reading and math scores drop in the sixth grade in Richmond, moreso for the ED group.

We know that increasing ED correlates with decreasing overall division pass rates.  Could it be that increasing ED populations (Richmond was 64% ED in the 2017 school year) also pull down the score changes? 

Here are the 2017 Division reading pass rate changes from fifth to sixth grade, for both the ED and non-ED populations, plotted vs. the division % ED students.

image

Richmond is the yellow points.

The R-squared values of the fitted lines tell us that the division pass rate changes for both the ED and non-ED groups are essentially uncorrelated with the %ED.  Indeed, if there were a correlation, it would seem to falsify the hypothesis: The slope is positive, which implies increasing sixth grade pass rates, relative to the fifth grade scores.  And, in fact, many divisions enjoy pass nice rate increases from fifth to sixth grades. 

The math scores tell the same story, albeit with more increased pass rates than decreased.

image

To the extent that ED correlates with home environment, these data falsify the hypothesis.

As well, notice that most of the divisions with higher ED populations than Richmond enjoy score increases from grade 5 to 6 (and none of those divisions suffers a decrease as large as Richmond’s).

image

image

Which leaves us to wonder whether those unusually large drops in Richmond are the product of cheating in the elementary schools or awful teaching in the middle schools.  Or both.

Ambiguous News in Richmond

The Times-Dispatch this morning reports that Richmond has (at last) decided to do something about its ongoing educational disaster.

The RT-D quotes interim Chief Academic Officer Victoria Oakley: “The bulk of our work needs to be with these partially warned and the schools that are in denied status to move them to full accreditation.”

We might hope they would revise that statement to “The bulk of our work needs to be with these partially warned and the schools that are in denied status to move them to full accreditation.”

If Richmond tackles the hard problem – the schools whose repeated failures have landed them in the accreditation basement – they will necessarily solve the easier problems at the same time.  If they attack the spectrum, there will be a temptation to focus on the easy wins.  That could impair the effort in the schools where it is most needed – and where nearly half of Richmond’s students languish in failed schools.

image

(2017 data here and here)

Please recall that they are dealing, in the worst case, with this appalling situation:

http://calaf.org/wp-content/uploads/2017/08/image-142.pnghttp://calaf.org/wp-content/uploads/2017/08/image-143.png

The (perhaps) good news in the story:

[The] work involves meeting with school principals to make individual plans and see where intervention or professional development is needed. Schools are creating progress plans that will be tracked monthly.

There are at least two ways to translate that bureaucratese:

  • If it means “hold our principals accountable for retraining or firing the ineffective teachers,” it may be the harbinger of a genuine effort to fix even our worst schools.
  • If it means more bloated plans, it sentences thousands more of Richmond’s schoolchildren to ineffective teaching on into an indefinite future.

Johnny Can’t Read But Greene Is Accredited Anyhow

This year, VDOEadjusted” the ESH Greene Elementary reading SOL from 51 to 81, the math from 58 to 84, and pronounced that school “Fully Accredited.”  Indeed, Greene has been fully accredited for at least the past eight years.

Let’s take a look back and see if these helpful pass rate “adjustments” have a history.

For a start, English.

NOTE: In the period of the data here, they administered a fifth grade writing test from 2011 to 2013.  The “English” SOL is some kind of average of that with the Grade 3-5 reading scores.  The numbers below are the average of the grade 3-5 reading and grade 5 writing for the 2011-13 period.  Given that the writing score covers only one grade, the actual averages probably are closer to the reading numbers.

image

For the purposes of the present analysis, the 2013-14 numbers are so appalling appallingly (oops!) low that those differences are beside the point.

Here, then, are the English pass rates and the “adjusted” values.

image

The orange line is the 75% accreditation benchmark.

For the past five years, the “adjustments” have boosted Green into accreditation.  This year the boost was thirty(!) points.

The math situation is slightly less egregious: The boost this year was only twenty-six points.

image

Courtesy of this falsification by our “education” establishment, Greene has remained fully accredited to the present.

HOWEVER

This official mendacity gives Greene bragging rights while failing to teach nearly half its students to read or reckon.  Unfortunately, that’s only the surface problem.

More fundamentally:  Accreditation — or lack of it — is meaningless.

In theory, a school that loses accreditation is in violation of the Standards of Quality and the Board of “Education” has the authority to compel the local school board to bring that school into compliance.  In practice, the Board does not know how (Sept. 21, 2016 video starting at 1:48) to do that.

The poster child in this respect is Petersburg, which has been operating under Memoranda of Understanding with the Board since April 2004 and still can’t teach 40% of its schoolchildren how to read.

image

In light of

  • the fictitious accreditation numbers,
  • the Board’s ineffective “help,”
  • the Board’s total failure to exercise its authority, and
  • the Board’s admission that it does not know how to fix broken schools

the accreditation process is a shameful sham.

Your tax dollars at “work.”

—————————-

Added Note:

The “adjustment” here appears to lie with the “English Learner” or “Limited English Proficiency” population.  (The VDOE Web site seems to use those terms interchangeably.)

To start out, the local school gets to decide which students are LEP (or EL).  For the reasons you’ll see below, I’ll bet you a #2 lead pencil that every student – except for the bright ones – who knows how to pronounce “señor” gets classified as LEP.

The LEP population affects the scoring in two ways.

First, “LEP students who have attended school in the United States for less than 12 months may receive a one – time exemption for the SOL Reading test in grades 3 through 8.”

So we get a one-time SOL score boost for any new, LEP student.

Then, to the point here, “The scores of LEP students enrolled in Virginia public schools fewer than 11 semesters may be excluded from the accreditation rating calculations.”

So LEP students who start here in kindergarten don’t count against accreditation until they’ve had six years to learn our Mother Tongue.  And those who start in the first grade or later need not be taught English; they can’t hurt the accreditation rating. 

Can’t you hear the mantra: “Teach the bright ones; forget the rest.”

Greene has a large population of “EL” students.  The Fall, 2017 report shows:

image

For sure, the Greene data tell us many of those kids are not being taught English.  Or math, which ought to be the same in Spanish as in English.

So the accreditation system accomplishes three things at Greene:

  • It produces accreditation numbers that are unrelated to how well the school is teaching,
  • It tells the school to classify all but the brightest immigrant students as LEP, whether those kids are fluent in English or not, and
  • It encourages the school to forget about the LEP kids; they can’t affect the accreditation even if they don’t learn a thing.

Your tax dollars at “work.”

Accreditation Inflation

The initial accreditation results tell us that 18 of 44 Richmond schools were fully accredited this year.  There remain 26 schools that, even by the highly flexible standards of the Board of “Education,” have a problem.

A closer look at the data shows a situation that is worse – in a couple of cases, much worse – than even those numbers suggest.

ESH Greene Elementary

Greene is fully accredited this year, despite a sad history of decline.

image

Recall that the accreditation benchmark is 75 for English, 70 for the other subjects.  Here we see Greene underwater in all five subjects.  Indeed, by any honest measure, Green would have run out its four years of failure last year.

But all that was before the “adjustments” provided some remarkable enhancements.  Here are this year’s results:

image

49.5% of the Greene students flunked the reading SOL but the “adjustments” produced a score six points above the 75% benchmark.  (They stopped reporting writing in elementary school three years ago so we are spared the ambiguity in averaging the reading and writing pass rates to get to the English datum.)

Greene enjoyed similar, if smaller, “adjustments” in the other subjects; the school then “met” the benchmarks in all subjects except Science, where a ten point boost still left it thirteen points short.  The school remains fully accredited.  Never mind it plainly is failing to properly serve its students.

There are good, old English words to describe this situation.  Most of them can’t be used in polite company.

Seven other Richmond schools were boosted into accreditation in one or more subjects.  Fortunately, none as dramatically as Greene.

Franklin Military Academy

Franklin has done quite well recently, except for a problem with math.

image

The “adjustments” this year camouflaged that problem.

image


Linwood Holton Elementary

Holton’s picture is similar to Franklin’s.

image

image


JL Francis Elementary

Francis has a persistent problem with reading; this year its SOLs dipped in math and science. 

image

The “adjustments” this year cured the latter two problems.

image


Miles Jones Elementary

In recent times, Jones has done well in history but has been borderline or below in reading, math, and science.

image

The “adjustments” this year cured all those little problems.

image


Southampton Elementary

Southampton had been improving but slid this year below the English and math benchmarks.

image

The “adjustments” fixed that.

image


Bellevue Elementary

Bellevue enjoyed the “adjustment” in math but not in reading.

image

image


Richmond Community High

Community usually does quite well.  This year, however, its math performance slipped below the benchmark.

image

The “adjustment” took care of that.

image


Comment
Rant

VDOE writes the SOL tests.  They can boost the pass rates and accreditation rates simply by making the tests easier.  Yet they indulge in this opaque process to produce meaningless numbers that favor some schools over others.

Moreover, they do not adjust the scores for the one factor that they measure and that we know affects the pass rates: Economic Disadvantage.  Indeed, they have abandoned (pdf at p.102) their measure of academic progress, the SGP, that is independent of economic status.

(Their excuse for dropping the SGP is, they say (pdf at p.102), that they can’t calculate it until all the data are in at the end of the summer.  I don’t think they were too stupid to know that when they started with the SGP; I think they are lying: The SGP gave intolerable amounts of accurate, comparable information about teacher/school/division performance.)

And then we have Petersburg that has managed to remain without accreditation for at least fourteen years, despite all the helpful “adjustments.”  The Board of “Education” has at last figured out what to do about that: They are going to change the accreditation system to make the process still more byzantine and to make it easier for a school to remain accredited.

Your tax dollars at “work.”