Pell Plus

We have seen that, among Virginia’s public, 4-year college programs, the graduation rate of Pell grantees correlates strongly with the overall graduation rate of the school.  We also have seen that (federal) Pell and (state) Commonwealth award holders on average graduate at lower rates than students who receive no financial aid.  As well, the data show that students receiving other forms of support graduate, on average, at higher rates than students with no aid.

SCHEV has some more data on this subject. 

Here we look at the 4-, 5-, and 6-year cohort graduation rates of the 2010-11 cohort of first time in college students at our Big Hitter universities.  These data count graduations anywhere, not just at the starting institutions.

The All Students rates look like this:

image

The rates for students who do not receive any financial support are similar but mostly lower.

image

The differences between the two rates show some considerable differences between schools (and emphasize that averages do not tell the whole story).

image

The Pell grantees graduate at still lower rates (UVa excepted).

image

Or, in terms of the Pell rate vs. the no-support rate:

image

Here we see Pell grantees underperforming the no-support group except in the fifth and sixth years at THE University.  It seems that UVa’s selection process works even better for Pell students than for the students who pay their own way.  VCU is another story.

The other group we have seen to underperform on average are the Commonwealth Award grantees.

image

UVa and W&M report no grantees (or at least fewer then ten; see the SCHEV suppression rules).  Tech and Mason outperform here; VCU does not.

The athletic awards show a much different pattern.

image

Those are large differences.  At six years, VCU graduated 13.7% more of its athletes than of its no-support students.  Tech, 18.4% fewer.

BTW: At five and six years, W&M graduated 100% of the supported athletes.  Consistent with its high overall rate, UVa graduated 90.2% at five years, 92% at six.

image

Next, here are the results for the other grant programs whose grantees, averaged over Virginia’s 4-year programs, outperformed the no-support group:

Perkins (Federal program for students enrolled in career & technical ed. programs):

image

PLUS loans:

image

Subsidized and unsubsidized Stafford loans:

image

image

Tuition waiver programs (mostly for employees and families; some for “older” citizens):

image

SCHEV-operated VGAP awards:

image

Federal work-study:

image

Notice the relatively high rate of 4-year degrees (and the widespread overperformance) among the work-study students. 

Doubtless there are multiple factors driving these results.  We might expect those to include the granting process, the selectivity of the school, the pool from which a school draws, and probably other factors.

Nonetheless these data suggest some tentative conclusions:

  • UVa and W&M are skimming the cream from the population of students who receive financial support;
  • As the earlier data suggested, the SCHEV-run VGAP awards are much more effective, in most cases, than the school-run Commonwealth awards;
  • Some schools run counter to the average underperformance of the Pell and Commonwealth grantees (e.g., UVa on Pell; Tech and Mason on Commonwealth); and
  • VCU’s relatively high graduation rate of athletes might suggest either careful selection and nurturing or corrupt grading practices.  It would be good to know which.

Diploma Inflation

The standard diploma requires five “verified credits” (i.e., passing the course plus passing the End of Course (EOC) SOLs or approved substitute tests) in English, math, laboratory science, and history & social science, plus one further verified credit, presumably in one of those areas.   The advanced diploma requires three further verified credits.

We have seen that the 4-year cohort graduation rate statewide has risen in recent years at the same time that the EOC SOL pass rates have generally declined.

image

The five-year graduation rates run a bit higher but show the same pattern (except, of course, that the data end a year earlier).

image

Well, that’s the state.  Let’s look at some divisions.  All the graphs below show the 4-year cohort rates.

Let’s start with Fairfax where, for sure, all the students are above average.

image

Hmmm. Excellent numbers but, again, declining pass rates and increasing diploma rates.  That’s easier to see if we just look at the subject average pass rate and the total diploma rate.

image

Loudoun looks much the same.

image

image

Then we have Richmond and peers.  (Notice the rate of standard diplomas higher than the advanced rate, contra the NoVa jurisdictions and state averages.)

image

image

image

image

image

image

image

image

Finally, because I sometimes have readers there, Charles City and Lynchburg.

image

image

image

image

There are some interesting differences here, particularly the vast differences between the NoVa jurisdictions and the old cities and the lower rates of advanced diplomas in those cities.

The variability of the small numbers in Charles City makes conclusions there problematic but otherwise the underlying theme is constant: Decreasing pass rates with increasing graduation rates.

These data surely allow VBOE to brag on the increasing graduation rates.  Whether they can brag that the kids are learning more is a question these data do not answer.

And, for sure, these data confound the notion that decreasing pass rates should lead to decreasing graduation rates.

Diploma Inflation?

The estimable Jim Bacon the other day raised the question whether Virginia’s increasing graduation rates might be related to grade inflation.

We have some data on a nearly-related subject:  Recall that the standard diploma requires five “verified credits” (i.e., passing the course plus passing End of Course (“EOC”) SOLs or approved substitute tests) in English, math, laboratory science, and history & social science, (Added note): plus one in a subject of the student’s choice (presumably in one of those five areas) (Hat Tip, the illustrious Chuck Pyle).  The advanced diploma requires three further verified credits.

The VDOE Web site has the Virginia average 4-year cohort graduation rates (back to 2008) along with the End of Course pass rates.  Here are the diploma rates along with the reading pass rates:

image

Hmmm.  Looks like the pass rate declined slowly through 2012, dropped a bit with the new (harder) tests in 2013, and remained about flat afterward.  During this time, the graduation rate with standard diplomas remained close to 35% while the rate of advanced diplomas rose from 44% to 52%, both with no dip in 2013.

Since Excel already has the data, let’s look at the correlations.

image

image

The writing tests tell much the same story.

image

image

image

That’s English.  How about math?

image

Here the new tests in 2012 had a large effect on the pass rate but the advanced diploma rate still rose unimpeded.  The (negative!) correlations are substantial:

image

image

Well, how about History & Social Science?

image

image

image

Finally, science.

image

image

image

Whatever is going on here – and the process is so byzantine that an outsider might despair to understand it – it is clear that the average graduation rate, especially of advanced diplomas, is not constrained by the EOC pass rates.  If anything, the graduation rates and pass rates are going in different directions.

Bacon mentions grade inflation.  This looks like diploma inflation somewhere outside the verified credit process.

Note added July 26: Here is a summary graph with all the data:

image

More Money for What?

Table 13 in the 2016 Superintendent’s Annual Report sets out the fiscal year’s disbursements by school division. 

The table reports day school disbursements (administration, instruction, attendance & health, pupil transportation, and O&M) plus disbursements for food service, summer school, adult education, pre-K, “other” educational, facilities, debt service, and contingency reserve.  See the footnotes to Table 13 for the details about these categories.  The numbers below omit facilities, debt service, and contingency reserve.

Taking the division totals, and dividing by the year-end Average Daily Membership (“ADM”), we see the following distribution:

image

Richmond is one of the three divisions at $15,500.

Note: Excel’s histogram analysis rounds up, so Richmond’s $15,052, Charles City’s $15,237, and Franklin City’s $15,317 all get reported at $15,500.

Looking at Richmond, the division average, and the peer cities of Hampton, Newport News, and Norfolk, we see:

image

Richmond’s $3,051 excess over the state average, times our ADM of 1,253.51 21,826.00, (Oops!  Thanks Jeremy!) gives an excess disbursement of $66.6 million.

Plotting the division average reading pass rates v. the disbursements gives the following graph:

image

Richmond is the gold square.  The red diamonds, from the left, are Hampton, Newport News, and Norfolk.

The least squares fitted line suggests that pass rates decrease with increasing division disbursements but the R-squared tells us that the two variables are uncorrelated.

Here is the same graph for the math SOLs:

image

The next time somebody starts claiming that Richmond needs more money, it might be useful to ask what they are doing with the excess $66.6 million they already are spending to achieve the lowest reading and second-lowest math pass rates in Virginia.

MOU Zoo

According to VBOE, 27 of 44 (61%) of Richmond’s schools are not fully accredited.

This hardly comes as a surprise to VBOE: They’ve had the SOL data since last summer. 

As they report, last July our (soon to be ex-) Superintendent “indicated” that a division-level academic review was in order and in November VBOE voted (pdf @ p.365) to approve that review.  Yet the only progress to date is a “First Review of Division-Level Memorandum of Understanding for Richmond City Public Schools,” on the agenda for the June 22 VBOE meeting.

If VBOE had been sweating through a massive restructuring of the Richmond school system, this delay (while another year’s worth of kids are subject to damage by Richmond’s awful schools) might make sense.  Unfortunately, the only product of this foot-dragging is a meaningless bureaucratic mishmash – a “Memorandum of Understanding” that does nothing but create busywork and a “rough draft” plan that is not a plan.

The MOU

In bureaucratese, “Memorandum of Understanding” is “MOU.”

You might reasonably ask what an MOU is. 

Well, whatever it is, it’s not a statute, regulation, or contract, or an enforcement order authorized by statute, so it’s not something that could be enforced. 

That said, let’s read this one and see what it does.  Or doesn’t do.

The MOU tells us in seven paragraphs that VBOE or VDOE will:

  1. coordinate . . . to provide technical assistance;
  2. meet with the RPS Board “President” (our School Board calls her its “Chair”);
  3. meet with the RPS Superintendent;
  4. provide “oversight over processes, procedures, and strategies” to include approval for expenditures of state or federal funds;
  5. “work closely” with RPS personnel and (redundantly) approve expenditures;
  6. (redundantly) provide “oversight over processes, procedures, and strategies;” and
  7. modify the MOU at will.

There we have six unquantifiable busywork provisions that are quite silent as to any result beyond a major opportunity for delaying financial processes.  That paragraph 7, however, says something: The MOU is not an agreement; it is some kind of (empty) bureaucratic fiat.

Well, perhaps RPS will be doing something useful.  Let’s see.

Richmond’s eleven paragraphs say they will:

  1. tell VBOE who the top three candidates are before they hire a Superintendent (more on this below);
  2. meet with VDOE;
  3. give VDOE approval of expenditures (If this were enforceable, it would create a huge bureaucratic burden for VDOE and give it control of RPS.  It is not enforceable.  It looks mostly to be a mechanism for delaying RPS decisions to spend money.);
  4. consult with VDOE about instruction and staff development;
  5. consult with VDOE about human resources et al.;
  6. create a corrective action plan (that they haven’t managed to create in the past year);
  7. cause the RPS Superintendent to keep the RPS Board updated;
  8. require RPS people to “participate” in technical assistance and professional development, as specified by VDOE;
  9. appear before VBOE to report;
  10. send the Board and Superintendent to professional development training; and
  11. permit a VDOE bureaucrat to sit on the RPS Board ex officio if RPS is not fully accredited after eight years(!).

Nope.  More busywork.

Paragraph 7 is particularly instructive: If the Super does not keep the Board updated, you’d think they’d fire the Super.  VBOE’s intrusion to this level suggests overwhelming busybodyness at VBOE.  Or sublime dysfunction in Richmond.  Or both.

Deeper Dive

Let’s take a more detailed look at the buns that enclose this nothingburger.

Paragraph 1 provides:

Should a vacancy occur in the position of Division Superintendent, the Richmond City School Board will provide the Superintendent of Public Instruction and the President of the Virginia Board of Education the names and credentials of its top three finalists to fill a vacancy of Division Superintendent or Interim Superintendent at least 5 business days prior to making an offer to the preferred candidate. The credentials of applicants must include experience in leading successful school and division turnaround efforts as evidenced by a multi-year trajectory of improved student achievement outcomes on the Virginia Standards of Learning tests or comparable state-mandated assessments in school divisions outside of Virginia.

The first clause is conditioned on a vacancy we know will occur on July 1, so we get to wonder why it is there.  Indeed, the sentence is quite clear without that clause, so the clause is doubly unnecessary.

(Are you beginning to notice a pattern of redundant nonsense here?)

This paragraph doesn’t tell us what VBOE might do if they object to a candidate and RPS hires her anyhow.  Indeed, there is nothing VBOE can do:

  • VBOE can enforce (presumably by injunction) Title 22.1, but the MOU is not part of that Title.
  • VBOE can sue to enforce the Standards of Quality, but those do not require a turnaround specialist as superintendent.

Presumably VBOE could sue to enforce its regulations regarding division superintendents (here and here), but those regulations are silent as to turnaround experience.  And the MOU is not a regulation or order, so the “turnaround” requirement there is meaningless.

Indeed, this is close kin to the demands for approval of RPS expenditures: Intrusive busywork that is not authorized by law.

Turning to the other end of the sandwich, Paragraph 11 in the Richmond section:

The Richmond City School Board will permit [a VDOE] selected representative to meet with the local board in an ex-officio, non-voting, member capacity should the division fail to have all of its schools Fully Accredited by the beginning of the 2025-2026 school year.

Aside from being quite unenforceable, this provision is ridiculous: If having a State bureaucrat sit on the RPS Board ex officio might accomplish anything, VBOE should demand it now.  The ongoing, outrageous harm to Richmond schoolkids is too high a price for delaying action at all, much less until 2025.

But, of course, eight years is a good number for VBOE: All of the members will have been replaced by then.

The Tell

Paragraph 11 is important for what it does not say: Richmond is in gross violation of the Standards of Quality and if RPS don’t do what VBOE wants, VBOE will sue them.

To the same end, the last section of the MOU, “Additional Consequences for Non-Compliance” quotes a 2016 statute that authorizes VBOE to withhold payment of At-Risk Add-On funds if a local Board fails or refuses to satisfy VBOE in a division-level review.

That statute is part of the Acts of Assembly and is printed for the World in the Code; quoting it here does not add anything.  Presumably this is a threat. 

Probably an empty threat:  Withholding funds from a School Board that already is harming too many of the students in its charge would be counterproductive. 

More to the point, as paragraph 11 warned us, VBOE does not say, “If you haven’t fixed those schools by date x, we’ll sue you.”  That is because VBOE does not know how to fix Richmond’s broken school system (Sept. 21, 2016 video starting at 1:48).  They don’t know what to tell a judge that Richmond should be made to do, so they don’t even contemplate exercising their authority to sue.

Your tax dollars at “work.”

For Better Or Pell

We have seen that, among the Virginia public, 4-year college programs, the graduation rate of Pell grantees correlates strongly with the overall graduation rate of the school.

SCHEV has some more data on this subject.  Here, for the public, four-year institutions, for the entering student year 2010-11, are the four-, five-, and six-year graduation rates at the college of entry of first time in college, full time, students with and without Pell grants, grouped by the number of credits attempted in the first term.

image

The green curves are for students without Pell grants.  Those attempting 15 to 17.5 credits in the first term (blue diamonds) showed a 58% 4-year rate, while the 5- and 6-year rates rose to 76% each.  Those attempting 18 or more credits (orange diamonds) achieved slightly lower rates while those attempting 12 to 14.5 credits (yellow diamonds) graduated at rates about ten percent lower.

The Pell grant population are the gray curves with the same codes for the data point colors. 

Notice:

  • For all six groups, the rates did not increase much after the fifth year;
  • For both Pell and non-Pell, the students attempting 14.5 or fewer hours in their first terms graduated at about 10% lower rates than their peers who attempted 15 to 17.5 hour loads;
  • In both groups, students attempting still heavier loads — 18 hours or more — graduated less often than those attempting 15-17.5 but still more often than the 12-14.5 tranche. 

That last is only partially consistent with the notion that A students underestimate their abilities while D and F students overestimate theirs.  Something else is at work here.

In any case, the bottom line remains the same as in the earlier data: Pell grants are subsidizing a lot of failure.

Go to Pell

After the NYT raised the issue of Pell grant percentages at elite universities, this blog and (more cogently, methinks) the estimable Jim Bacon commented on those and other Pell issues.

To extend the conversation, I’ve revisited the Virginia Pell and graduation rate data.  Here, to start, is a plot of the 2010-11, first-time-in-college, six-year cohort graduation rates of students with Pell grants at entry vs. all students for the public, four-year Virginia programs.

image

The correlation here is excellent: Schools that graduate more of their students also graduate more of their Pell students.

At the same time, the average Pell graduation rate is 8.3% below the overall rate, and the Pell rate increases slightly more slowly than the overall rate (98% as fast).

The fitted line, above, gives us a measure of the average (well, the the best fit in terms of the minimized sum of squares of the residuals) Pell performance of these schools vs. the overall school rate.  Plotting the differences between actual Pell rates and those calculated from the fitted line gives a measure of relative Pell performance among these Virginia schools.

image

VMI, Longwood, and VSU turned in outstanding Pell graduation rates, compared to their overall rates.  UVa (excuse me, THE UNIVERSITY), which the Times maligned, did a better than average job of graduating the Pell students it admitted. 

And notice that UVa and W&M are improving on 90%+ graduation rates, which is not an easy thing to do.

These data don’t tell us whether the superior performance comes from more careful admissions, better support, or something else, either alone or in combination.  But, for sure, these schools are doing something right, or at least better than their Virginia colleagues.

At the risk of lese majesty against the Times, I’ll say that UVa is not to be criticized for small Pell populations; it is to be praised for a superior job of graduating its Pell students. 

BTW: Kiplinger says that, as of 2014, UVa was one of only two public colleges to meet 100% of its students’ financial need. 

Then we have James Madison, VCU, Old Dominion, Norfolk State, and Tech all performing worse than their overall graduation rates would predict. 

Something needs fixing there.

Graduation and Not, With and Without Financial Aid

We have seen that, averaged over Virginia’s public, four-year programs, Pell and Commonwealth Award grantees have underperformed considerably in terms of graduation rates.  The military survivor/dependents students underperformed at four years but led the pack at six.  Perkins students (career and technical grants) outperformed at four and five years but were slightly below the military rate at six.  The SCHEV-administered VGAP students underperformed the no-support group at four years and outperformed at five and six; they considerably outperformed the school-administered Commonwealth Award group at all levels.

The SCHEV reports also provide data by institution; I have taken a look at the five research universities.

Note: Data are for the 2010-11 first-time-in-college cohort; the graduation rates include degrees at any institution.  Graphs below report the differences between the group that received the specified support at the beginning of college and the group receiving no support at the same university.

SCHEV did not report military survivor, etc. data for any of the five universities so each had nine or fewer of those students (SCHEV does not report data for groups <10). 

As to the Perkins career and technical grantees, this is the picture:

image

Note: No Perkins data for George Mason.

Second Note: The no-support rates at these universities are different so we need to view the graph with some care:

image

Indeed, a 1% improvement over the 94.8% 6-year at UVa or the 95.1% at W&M is a Big Deal.  Similarly Tech at 90.2%.  The two urban U’s, not so much.  Negative differences indicate underperformance vs. the no-support group and any such, methinks, raises a yellow, if not red, flag.

One might wonder whether these research universities would exhibit a strong draw for career and technical ed. students; in any case, W&M outperforms (spectacularly) (again, relative to its own students not receiving any aid).

BTW: Those 5- and 6-year Perkins numbers at W&M both are 102.9%.  You can ask SCHEV how a cohort pass rate can go over 100%.

Turning to the other end of the spectrum, the Pell grantees:

image

The New York Times pinged UVa for not taking a lot of Pell students.  They did not deign to notice that THE UNIVERSITY does a good job of graduating those Pell grantees that it does admit.  Indeed, that +2.4% at six years comes from a 97.2% pass rate for the Pell grantees.

VCU’s Pell grantees are quite another story.

SCHEV does not report Commonwealth Awards data for UVa and W&M.  Students with that support do OK at Tech and Mason, not so OK at VCU.

image

Next, the athletes.

image

VCU shines here, graduating 22% more students at five years with that aid than with none, 14% more at six years (87.1% in both cases).

Finally, here are the results for the other forms of financial support, starting with PLUS loans.

image

That 6-year UVa rate is 100.7%.  Good work if you can get it.

Stafford loans, subsidized:

image

Stafford loans, unsubsidized.

image

Tuition Waiver for faculty and staff, e.g., at VCU, and some waivers for mature citizens:

image

The Virginia Guaranteed Assistance Program:

image

At five and six years, this program looks to be a general success.

And finally, Work-Study:

image

Another success, esp. at four years in four cases.

Now let’s all read Part 4 of Jim Bacon’s series on higher-ed. accountability (due out approximately tomorrow) to see if his numbers make all this unnecessary.

Worse Than Pell!

We have seen that larger percentages of Pell grants are associated with lower graduation rates in Virginia’s four-year institutions. 

SCHEV has some more granular data:  Here, to sample that pool, are the 4-, 5-, and 6-year completion rates for the 2010-11, first-time-in-college cohort in Virginia’s 4-year programs.  The data count degrees completed at any institution.

image

Let’s look at each of those groups (and its six-year rate):

  • Military survivor/dependent awards: 86.9%.   Tuition and fees waived at Virginia public institutions for survivors and dependents of certain military service members.
  • Perkins Grants: 86.3%.  Federal program for students enrolled in Career & Technical Education programs. 
  • PLUS Loans: 83.2%.  Loans from US Education Dept. toward cost of attendance, minus other financial aid.
  • Athletic awards: 83.1%.  (Go figure!)
  • Stafford Loans: 82.9% unsubsidized; 81.5% subsidized.  This is the federal student load program.  Subsidized loans, available to students with demonstrated financial need; the feds pay the interest while the student is in school.  Borrowers are responsible for all the interest on unsubsidized loans.
  • Work-Study: 81.4%.  Program for federal + institutional support to provide part time employment for needy students.
  • VGAP award: 81.4%.  Virginia Guaranteed Assistance Program for Virginia residents, administered by SCHEV.
  • Tuition Waiver: 77.9%.  Mostly institutional programs for faculty and employees.  E.g., VCU (note: my browser complains of a bad site certificate here), UVa.  But see mature (they say “older”) citizens.
  • No financial aid: 77.6%.
  • Pell grants: 69.1%. 
  • Commonwealth Award: 65.0%.  State funds for undergraduates with financial need; appropriated directly to state-supported institutions.

There are several surprises here.  One that jumps out is that Commonwealth Awards are subsidizing failure at a rate even higher than the Pell grants.

The military awards are not a surprise.  In my experience, at least, veterans are different from, and much more serious than, most other students.  Doubtless veterans’ spouses and dependents partake of that difference. 

In Virginia’s 4-year programs, both scholarship athletes and students with federal loans graduate at higher rates than students with no financial aid. 

The SCHEV-administered VGAP graduation rate shines in comparison to the Pell and the school-administered Commonwealth Award rates.  Perhaps it’s time to reallocate that Commonwealth Awards money.

And the major takeaway:  Perkins!  We need more plumbers, electricians, network admins, and auto mechanics and this program looks to be effective in getting them educated.  Indeed, the 4-year rate is exceptional.  These data suggest some places where it might be wise to reallocate funds to Perkins grants.