Dollars But Not Scholars

I earlier demonstrated that division expenditure per student does not correlate with division SOL pass rate in reading or math.

Jim Weigand of Lynchburg points out that VDOE prepares an annual report on each division’s “expenditures and appropriations designated to meet their required local effort in support of the Standards of Quality.”  The 2015 report is not yet out so the available data are 2014.  Jim juxtaposed those data the excess expenditure reported there with the 2014 five-subject (Reading, Writing, History & SS, Math, and Science) average pass rate.  Here is the result:

image

The fitted line might suggest that pass rate increases by 2.7% for a doubling of the required local effort but the R2 tells us that the correlation is minuscule.

The gold square is Richmond.  For Jim’s interest, the yellow diamond is Lynchburg.  The red diamonds are, from the left Hampton, Norfolk, and Newport News.

The average excess RLE is 84.  The Big Spender out there at 221% excess RLE with the 63% pass rate is Sussex; the next Big Spender at 218% excess RLE with the 91% pass rate is West Point.

Here are the data.  Accomack is missing for lack of a spending report.

5 Subject Avg. SOL Rank Excess RLE Rank
Albemarle County  80.17% 25 140.25% 12
Alexandria City  68.76% 109 183.58% 4
Alleghany County  70.25% 104 180.40% 5
Amelia County  73.02% 88 44.58% 106
Amherst County  75.47% 67 94.26% 51
Appomattox County  77.00% 49 15.34% 127
Arlington County  84.12% 10 193.87% 3
Augusta County  76.36% 54 77.48% 67
Bath County  77.60% 43 118.81% 28
Bedford County  75.35% 70 87.24% 57
Bland County  75.21% 72 38.08% 112
Botetourt County  84.78% 6 132.86% 19
Bristol City  76.59% 23 44.94% 104
Brunswick County  63.99% 125 17.81% 126
Buchanan County  67.65% 111 73.69% 71
Buckingham County  71.86% 94 37.03% 113
Buena Vista City  64.64% 122 63.02% 91
Campbell County  75.97% 58 112.60% 30
Caroline County  71.78% 95 36.65% 114
Carroll County  76.01% 57 102.29% 39
Charles City County  77.73% 42 95.38% 49
Charlotte County  77.04% 48 34.84% 116
Charlottesville City  75.57% 64 154.45% 9
Chesapeake City  81.47% 18 114.57% 29
Chesterfield County  79.41% 31 82.30% 62
Clarke County  77.50% 45 101.54% 52
Colonial Beach  64.00% 124 64.95% 85
Colonial Heights City  79.54% 30 171.97% 7
Covington City  71.32% 97 152.31% 10
Craig County  74.41% 78 39.00% 110
Culpeper County  75.39% 69 60.11% 92
Cumberland County  66.77% 115 69.99% 77
Danville City  65.36% 120 88.81% 54
Dickenson County  74.09% 80 63.43% 89
Dinwiddie County  71.94% 93 70.52% 76
Essex County  63.21% 126 49.29% 100
Fairfax County  84.02% 11 127.86% 22
Falls Church City  91.10% 1 170.65% 8
Fauquier County  79.24% 33 112.23% 31
Floyd County  75.95% 60 45.87% 102
Fluvanna County  77.52% 44 65.86% 83
Franklin City  59.36% 131 129.24% 21
Franklin County  81.99% 17 64.35% 88
Frederick County  75.54% 65 124.29% 23
Fredericksburg City  74.36% 79 134.11% 18
Galax City  74.69% 75 70.74% 75
Giles County  76.71% 52 43.03% 108
Gloucester County  77.27% 47 98.21% 46
Goochland County  83.94% 12 59.77% 93
Grayson County  72.39% 90 38.09% 111
Greene County  73.62% 83 73.40% 73
Greensville County  67.45% 112 27.61% 121
Halifax County  67.78% 110 34.36% 117
Hampton City  70.83% 101 88.31% 56
Hanover County  84.36% 7 58.92% 94
Harrisonburg City  71.17% 100 102.25% 40
Henrico County  77.94% 39 69.50% 79
Henry County  74.00% 81 39.17% 109
Highland County  66.24% 117 23.29% 124
Hopewell City  67.25% 113 73.16% 74
Isle of Wight County  81.38% 19 68.85% 80
King and Queen County  70.30% 103 73.63% 72
King George County  77.88% 40 53.80% 98
King William County  78.66% 36 100.45% 44
Lancaster County  62.39% 128 77.00% 68
Lee County  74.90% 74 9.93% 130
Lexington City  83.32% 14 52.26% 99
Loudoun County  86.03% 4 138.33% 14
Louisa County  79.26% 32 69.69% 77
Lunenburg County  69.29% 108 24.07% 123
Lynchburg City  64.39% 123 103.34% 37
Madison County  72.18% 92 136.20% 16
Manassas City  71.38% 96 172.35% 6
Manassas Park City  71.20% 99 102.57% 38
Martinsville City  59.78% 130 111.14% 32
Mathews County  76.74% 51 58.44% 95
Mecklenburg County  66.65% 116 29.19% 119
Middlesex County  80.71% 23 35.60% 115
Montgomery County  78.83% 34 79.77% 66
Nelson County  78.53% 37 101.51% 43
New Kent County  80.90% 21 81.64% 64
Newport News City  66.85% 114 110.30% 33
Norfolk City  65.94% 118 90.52% 52
Northampton County  65.29% 121 31.46% 118
Northumberland County  77.28% 46 56.26% 96
Norton City  78.01% 38 47.40% 101
Nottoway County  70.15% 105 27.39% 122
Orange County  76.35% 56 63.27% 90
Page County  73.43% 85 64.59% 87
Patrick County  74.44% 77 11.02% 129
Petersburg City  57.64% 132 44.37% 107
Pittsylvania County  78.76% 35 22.77% 125
Poquoson City  86.21% 3 97.70% 47
Portsmouth City  69.40% 107 85.72% 58
Powhatan County  82.01% 16 107.65% 35
Prince Edward County  65.42% 119 95.62% 48
Prince George County  79.95% 27 45.02% 103
Prince William County  79.81% 29 98.56% 45
Pulaski County  74.92% 73 65.25% 84
Radford City  75.41% 68 83.96% 60
Rappahannock County  76.85% 50 76.17% 70
Richmond City  59.84% 129 90.30% 53
Richmond County  75.93% 61 76.54% 69
Roanoke City  71.21% 98 132.74% 20
Roanoke County  85.33% 5 103.62% 36
Rockbridge County  75.48% 66 82.02% 63
Rockingham County  79.99% 26 138.87% 13
Russell County  75.58% 63 29.03% 120
Salem City  83.35% 13 142.76% 11
Scott County  80.75% 22 13.33% 128
Shenandoah County  75.58% 62 84.78% 59
Smyth County  73.86% 82 44.66% 105
Southampton County  73.52% 84 68.22% 81
Spotsylvania County  76.35% 55 120.89% 26
Stafford County  81.24% 20 123.97% 24
Staunton City  73.13% 87 88.40% 55
Suffolk City  72.75% 89 66.41% 82
Surry County  73.16% 86 136.52% 15
Sussex County  62.89% 127 221.04% 1
Tazewell County  77.87% 41 8.84% 131
Virginia Beach City  79.85% 28 120.91% 25
Warren County  75.97% 59 83.74% 61
Washington County  80.71% 23 108.98% 34
Waynesboro City  70.40% 102 120.38% 27
West Point  90.97% 2 217.53% 2
Westmoreland County  69.96% 106 54.40% 97
Williamsburg-James City County  82.58% 15 94.36% 50
Winchester City  74.62% 76 134.37% 17
Wise County  84.27% 9 101.67% 41
Wythe County  75.25% 71 64.87% 86
York County  84.33% 8 80.51% 65

Middle School Mess

The VDOE database is glad to produce pass rates by grade. 

We must view these numbers with some caution.  The high school pass rates are boosted by inclusion of the Maggie Walker students who live in Richmond, albeit Walker is not a Richmond public school.  They still are giving the notorious VGLA to LEP students in grades 3-8 (albeit it’s now graded by the state, not the local schools). 

Here, then, are the pass rates for the 2015 reading tests for Richmond and the State. 

image

And here are the pass rates on the math tests.

image

We can simplify the pictures by taking the difference between the Richmond and state pass rates.

image

So we see the elementary schools underperforming, the middle schools failing their students miserably, and the high schools a very mixed bag.

For sure, if RPS elects to attack the worst of the awful, they’ll start with the middle schools and eleventh grade reading.

. . . Garbage Out

I’ve already discussed VDOE’s byzantine, opaque process for “adjusting” pass rates to calculate accreditation status.  So, without further comment, and for whatever these numbers may mean, here is the distribution of Full accreditations by division.

image

Richmond is the yellow bar.  The red bars are, from the left, Petersburg, Norfolk, Newport News, and Hampton.  The blue bar is the state average.

VDOE invented several forms of “nearly pass” categories this year.  Here is a list of the categories that appear in this year’s database, along with the abbreviations I had to use to fit the table below on the page. 

Accreditation Denied Denied
Conditionally Accredited (New Schools) New School
Fully Accredited Full
Partially Accredited: Approaching Benchmark-GCI Close, GCI
Partially Accredited: Approaching Benchmark-Pass Rate Close, Pass Rate
Partially Accredited: Improving School-Pass Rate Improving, Pass Rate
Partially Accredited: Warned School-Pass Rate Warned, Improved Pass Rate
To Be Determined TBD

The complete list is here and the press release explaining the new categories is here.

Finally, here is the table, sorted by division.

  Denied New School Full Close, GCI Close, Pass Rate Improving Pass Rate Warned, Improved Pass Rate TBD Total % Full
Accomack County     8     2 1   11 73%
Albemarle County     19   1   6   26 73%
Alexandria City 1   12     2 1   16 75%
Alleghany County     4       1   5 80%
Amelia County     1   1 1     3 33%
Amherst County     7     1 2   10 70%
Appomattox County     4           4 100%
Arlington County   1 31           32 97%
Augusta County     14   1 1 4   20 70%
Bath County     2       1   3 67%
Bedford County     12       6 1 19 63%
Bland County     1   1       2 50%
Botetourt County     10       1   11 91%
Bristol City     4       2   6 67%
Brunswick County     2     1 2   5 40%
Buchanan County     7   1     1 9 78%
Buckingham County     1   1   2   4 25%
Buena Vista City             1 3 4 0%
Campbell County     12         1 13 92%
Caroline County     3       2   5 60%
Carroll County     7       2   9 78%
Charles City County     1       1   2 50%
Charlotte County     4       1   5 80%
Charlottesville City     9           9 100%
Chesapeake City     34   2 5 4   45 76%
Chesterfield County     52   1 3 5   61 85%
Clarke County     3     1     4 75%
Colonial Beach     2           2 100%
Colonial Heights City     5           5 100%
Covington City     2   1       3 67%
Craig County     2           2 100%
Culpeper County     7   1 1 1   10 70%
Cumberland County             3   3 0%
Danville City     3   1 1 5 1 11 27%
Dickenson County   2 1     1 1   5 20%
Dinwiddie County     5       1 1 7 71%
Essex County           1 1 1 3 0%
Fairfax County     177 1 2 3 9   192 92%
Falls Church City     4           4 100%
Fauquier County     17       2   19 89%
Floyd County     5           5 100%
Fluvanna County     5           5 100%
Franklin City     1         2 3 33%
Franklin County     16           16 100%
Frederick County     12   1 1 3 1 18 67%
Fredericksburg City     4           4 100%
Galax City     3           3 100%
Giles County     4       1   5 80%
Gloucester County   1 7           8 88%
Goochland County     5           5 100%
Grayson County     6       1   7 86%
Greene County     2   1   2   5 40%
Greensville County     1     1 2   4 25%
Halifax County     4     2 3   9 44%
Hampton City     12   4   8 5 29 41%
Hanover County     23           23 100%
Harrisonburg City     6     1 1   8 75%
Henrico County 1   45     3 17 1 67 67%
Henry County     11   2   1   14 79%
Highland County     1     1     2 50%
Hopewell City     1       4   5 20%
Isle of Wight County     8       1   9 89%
King and Queen County     3           3 100%
King George County     4     1     5 80%
King William County     4           4 100%
Lancaster County         1   2   3 0%
Lee County     8       2   10 80%
Lexington City     2           2 100%
Loudoun County   1 83   1   1   86 97%
Louisa County     5     1     6 83%
Lunenburg County     1       3   4 25%
Lynchburg City     3   1 1 8 3 16 19%
Madison County     2       2   4 50%
Manassas City     6   1 1     8 75%
Manassas Park City     3       1   4 75%
Martinsville City           1 3   4 0%
Mathews County     3           3 100%
Mecklenburg County     3   3   1 1 8 38%
Middlesex County     3           3 100%
Montgomery County     18       1   19 95%
Nelson County     3       1   4 75%
New Kent County     4           4 100%
Newport News City 3   15   2 6 9 3 38 39%
Norfolk City 4 1 17   2 4 10 7 45 38%
Northampton County 1   1     2     4 25%
Northumberland County     2   1       3 67%
Norton City     2           2 100%
Nottoway County     1     3 2   6 17%
Orange County     9           9 100%
Page County     4     1 3   8 50%
Patrick County     6     1     7 86%
Petersburg City 1   1       3 2 7 14%
Pittsylvania County     16     1 1   18 89%
Poquoson City     4           4 100%
Portsmouth City     11     2 4 2 19 58%
Powhatan County     6           6 100%
Prince Edward County     1       2   3 33%
Prince George County     6       2   8 75%
Prince William County   1 80   4 1 1 1 88 91%
Pulaski County     7       1   8 88%
Radford City     4           4 100%
Rappahannock County     2           2 100%
Richmond City 2 2 17     3 14 7 45 38%
Richmond County     2           2 100%
Roanoke City     15   1 3 5   24 63%
Roanoke County     26           26 100%
Rockbridge County     5       1   6 83%
Rockingham County     22   1       23 96%
Russell County     11       1   12 92%
Salem City     6           6 100%
Scott County     13           13 100%
Shenandoah County     5     1 3   9 56%
Smyth County     12       1   13 92%
Southampton County     5         1 6 83%
Spotsylvania County     27     1 1   29 93%
Stafford County     30           30 100%
State 13 9 1414 1 46 76 215 49 1823 78%
Staunton City     3     1   1 5 60%
Suffolk City     11     3 4 1 19 58%
Surry County     2     1     3 67%
Sussex County     2   1       3 67%
Tazewell County     15           15 100%
Virginia Beach City     73   3 1 3 2 82 89%
Warren County     6   1 1     8 75%
Washington County     14   1       15 93%
Waynesboro City     2       4   6 33%
West Point     3           3 100%
Westmoreland County     1     2 1   4 25%
Williamsburg-James City County     15           15 100%
Winchester City     5       1   6 83%
Wise County     12           12 100%
Wythe County     11       1   12 92%
York County     19           19 100%

In the Accreditation Basement

VDOE has posted the 2016 Accreditation Ratings, based on the 2015 test scores.

I’ll have more to say later about VDOE’s manipulation of the Accreditation Ratings, to include the newly minted “junior is flunking but by less than before” ratings.  For now, here are the Richmond results.

image

image

It’s hard to know what all those “TBD” entries mean.  I’ll have a look at the pass rates and post them here, soon.  For sure, 38% fully accredited is not good news.

And, also for sure, Thompson was denied accreditation last year but the newly-minted Elkhardt-Thompson is getting a free pass.

Preliminary Graduation Data

VDOE is out today with a press release bragging on the increased On-Time graduation rate

The VDOE Web site was down today until suppertime; I will not have time to analyze the data until tomorrow.  Until then, here are some early data (“Actual” rate refers to the advanced+standard diploma rate; “on-time” includes counts of those diplomas plus the modified standard, special, and general achievement  diplomas):

image

image

How About Those Salaries?

As a further look at the effect of expenditures on performance, here are the 2015 division average reading pass rates v. the 2015 division average budgeted teacher salaries (the actual 2015 salary data won’t be available until around the first of the year).  Data for Accomack are missing for lack of a report to VDOE.

image

Richmond is the gold square; the blue circle is the statewide division average.

The fitted curve suggests that an additional $10,000 average salary is associated with a 2% increase in the pass rate but the R2 tells us that the two variables are essentially uncorrelated.

The math data paint a similar picture.

image

Of course, we know that increasing economic disadvantage of the student population is associated with lower pass rates.  We can account for the average effect by using the correlation between pass rate and economic disadvantage to normalize the pass rates, i.e., express the pass rates as percentages of the economic disadvantage trendline rates.   That produces these graphs:

image

image

Again, only minuscule correlations.  And the fitted curves, to the extent they mean anything, say “no benefit from the higher salaries.”

So it seems that the divisions that pay their teachers more do not get better SOL performance; they merely pay more for the performance they get.

Finally, here for two of my faithful readers (maybe the only two) are the last two graphs showing the results for Charles City (purple circle) and Lynchburg (red circle).

image

image

Data are posted here.

JLARC Punts

The Joint Legislative Audit and Review Commission (JLARC) has just published its draft report “Efficiency and Effectiveness of K-12 Spending.”  Unfortunately, that report does not even look carefully at where Virginia is spending its educational dollars, much less answer the (much harder) question of what we are getting for that money.

The Mandates

The General Assembly gave JLARC two decently clear mandates.

SJR328 (2013): JLARC shall

study the efficiency and effectiveness of elementary and secondary school spending in Virginia.  [It] shall (i) study the efficiency and effectiveness of elementary and secondary school spending in Virginia, including evaluating the findings from School Efficiency Reviews and assessing the extent to which recommendations have been implemented; (ii) compare to other states how and to what extent Virginia funds elementary and secondary education; and (iii) identify opportunities to improve the quality of education students receive in consideration of the funds spent.

2014 Appropriation Act, Item 30 (at p. 62 of the link): JLARC to examine virtual instruction, to include “effectiveness of  virtual schooling in terms of  student academic achievement outcomes on assessment tests and course completion or graduation rates.”

The “Study”

The result is a 112 page draft that ignores the instructions of the General Assembly. 

Of the nine recommendations, six talk about efficiency; half of the six deal with school busses; only one of the six deals with something that relates to education.  None tells us about the educational effectiveness of our school spending or how to improve it:

  1. Track teacher turnover.
  2. Provide facilities management expertise.
  3. Provide “guidance” regarding sharing information about facilities management best practices.
  4. Consider statewide contract for bus routing and monitoring software.
  5. Provide transportation management expertise.
  6. Assist with transportation management best practices.

As to virtual schooling, JLARC again avoids answering the question.  The three recommendations:

  1. Provide information about online schools.
  2. Estimate costs of online learning.
  3. Compare achievement of virtual v. physical schools

That last one is particularly rich: JLARC is recommending that VDOE do what the General Assembly told JLARC to do.

Cranky’s Conclusion

This study is a wordy waste of money.  It does not answer the questions posed by the General Assembly.  Instead, it raises a new question: Why are we paying JLARC to not do what it’s been told to do?

A Reader’s Conclusion (added on 9/17)

A reader suggests an alternate (and more pertinent) conclusion: Why are we paying JLARC not to do what it’s been told to do, when we already are paying VDOE that should be doing [what JLARC failed to do]?

New (Federal) College Data

USDOE has just posted a considerable trove of college data.

CAVEAT:  These data are mostly for students who received federal financial aid. 

  • “Average Annual Cost”: The average annual net price for federal financial aid recipients, after aid from the school, state, or federal government. For public schools, this is only the average cost for in-state students.
  • “Graduation Rate”: The graduation rate after six years for schools that award predominantly four-year degrees and after four years for all other schools. These rates are only for full-time students enrolled for the first time.
  • “Salary After Attending”: The median earnings of former students who received federal financial aid, at 10 years after entering the school.

My quick reading of the data does not disclose what fraction(s)of the student populations are represented here. 

With that warning, here is a look at the Virginia public and not-for-profit colleges.  First the graduation rates:

image

The winners there are UVa in red, W&L in yellow, and W&M in green.

Next, the median salary ten years out:

image

W&L, in yellow, is the big winner here.

Finally, a bang/buck calculation, ((Salary * Graduation Rate) / Average Cost):

image

Colors, as before, are UVa in red, W&L in yellow.

Here is the dataset, sorted by school name.

image

You might be interested in comparing these data with the results of the Brookings “value-added” study.

Anatomy of a Lousy Performance: SOLs by School

Here are the 2014 and 2015 Reading pass rates by Richmond school:

image

image

image

You may recall that the accreditation benchmark for reading is a 75% pass rate.  VDOE cooks the accreditation numbers so thoroughly that the 75% criterion may be interesting as a rule of thumb but it is meaningless as to which schools actually get accredited.  You’ll notice that none of the mainstream middle schools and far too few of the elementary schools made 75% this year.  Indeed, King went from unspeakably bad to worse, never mind anything to do with 75%.

For another, perhaps more useful, measure, the statewide average reading pass rate was 79.0.

Franklin has both middle and high school grades so I’ve included it in both lists, although its scores can’t be directly compared to either.

Carver continued its spectacular performance, leading among the (only) six elementary schools to beat 75%. 

Next the math data.  Recall that the accreditation criterion is 70%.  The state average pass rate this year was 79.4.

image

image

image

Notice the decreases at Open and Marshall, as well as the uniformly miserable pass rates of the middle schools.  Note the several elementary schools doing pretty well, led again by Carver. 

SOL v. Cost

Table 13 in the Superintendent’s Annual Report lists annual disbursements by division.  Unfortunately, we only have the 2014 data; the current data ordinarily don’t come out until the following Spring.

Deleting the facilities, debt, and contingency entries, and juxtaposing the resulting disbursement totals with the 2015 Reading SOL Pass rates, produces the following graph.

image

Richmond is the gold square.  The red diamonds are, from the left, Hampton, Newport News, and Norfolk.  Thus we see the comparable old, urban jurisdictions performing poorly at about average cost while Richmond’s reading performance is much worse at a much higher cost.

The datum up there at $11,127, 23% less expensive than Richmond, is West Point, with an 87.8% pass rate.

The R2 value of 2.3% tells us that, among the Virginia school divisions, reading performance and cost per student are essentially uncorrelated.

The math data paint a similar picture.

image

The division pass rates again fail to correlate with expenditure. 

The point up top ($11,127, 89.0%) is West Point, again.

These data say, quite clearly, that Richmond’s education establishment should stop whining about money and start educating the City’s children.