SGP V – Bad Data?

Turning back to the question of the appalling performance of Richmond’s middle schools, especially in the sixth grade, let’s take a look at the SGP average by teacher by year.  Here, for a start, are the fifth grade data, first for reading and then for math:

image

image

These graphs make a couple of points:

  • In 41% of the cases, a teacher taught the subject for only one year year of the three.  For the most part, however, the teachers in both subjects taught the same grade and subject for either two years (21% and 22% for reading and math, respectively) or all three years (38% and 37%).
  • The dip in the reading SGPs in 2013 corresponds to the new reading tests that year and Richmond’s failure to align its curriculum.  Otherwise, the year-to-year changes give a quick idea of the variability per teacher.  To put a number on that, the average standard deviation of the annual SGPs is 11 for both sets of data; doubtless that number would be smaller if Richmond had not failed to prepare for the new reading test in 2013.

The sixth grade data show an entirely different pattern.  Again, first for reading and then math:

image

image

First, of course, the SGPs are much lower than the fifth grade numbers.

Remarkably, only one teacher (#66291) taught one subject (math) at the 6th grade level for two years in a row.  At least that’s what this database says.  Thirty-seven teachers taught reading only one of the three years while five taught two of the three years.  For math, those numbers are twenty-four and six.  As the graphs show, none taught for all three.

The small number of teachers reported in 2013 looks strange.  It is strange.  See below.

The seventh grade patterns are similar, albeit the scores are a bit higher and we see a few more teachers reported to be repeating the same subject.  Again, reading and then math:

image

image

Finally, the eighth grade scores, again reading first:

image

image

Aside from the remarkable teacher turnover implied by these data, the number of Grade 6 to 8 reading and math teachers is unbelievably low in 2013 except, perhaps, for the 8th grade reading.  As well, the counts of SOL and SGP results in the VDOE SGP database for grades 6-8 in 2013 look to be anomalous:

image

image

We have nowhere to go to test the number of SGP reports.  In contrast, the (very nice) VDOE SOL database gives the participation counts for the same years and grades.  Here, then are the Richmond numbers of SOL results in the SGP database, expressed as percentages of the participation numbers reported for the same tests.  First, reading:

image

Then math:

image

Let’s first deal with the ridiculous datum: The “#DIV/0!” for 2014 6th grade math is there because VDOE reported zero participation in that test:

image

Looking beyond that obvious error: VDOE attached a sheet to the earlier download, admitting that the SGP database suppressed data for classes fewer than ten students and for students who transferred from one Virginia school to another during a school year.  They provided counts for the suppressed data by test, ranging from 1.6% to 7.3%, with an average of 4%.

Except for the 2012 Grade 6 math datum, where the SGP database reports SOL scores for 106% of the students who took the test, the numbers here are much lower than even a 7% suppression (low in elementary school, very low in middle school).  They are ridiculously low for grades 6-8 in 2013.

Doubtless that explains the remarkable teacher turnover reported for the SGP in grades 6-8 for 2013: Leave out the test data and the reports by teacher also will be left out.

We’ll have to try to get a correct SGP database and see how that affects the SGP analysis here and earlier.

Sigh.