Scores Follow National Trend | Eastern North Carolina Now

    Publisher's note: This article appeared on John Hood's daily column in the Carolina Journal, which, because of Author / Publisher Hood, is linked to the John Locke Foundation.

    RALEIGH - North Carolina schools got a mixed report from the latest independent tests, known as the National Assessment of Educational Progress (NAEP). During the coming political season, you should expect to hear a lot about those NAEP results. Unfortunately, you should not expect to hear a lot of useful information about them.

    Education is the single largest enterprise financed by state and local governments. Everyone recognizes that it is critical to a productive economy, to good citizenship, and to a healthy society. Everyone has an opinion about it. Everyone should.

    Those opinions, however, should not be used as filters to screen out inconvenient information. What social scientists call "confirmation bias" is a major problem in debates about education. Rather than scan for news that makes our "side" look better and the other "side" look worse, we should read voraciously, from many different sources, and withhold judgment until the information is carefully evaluated and considered.

    In the 2015 NAEP scores, both defenders and critics of Republican rule in Raleigh can find talking points to their liking. North Carolina's 8th-graders experienced four-point declines in both reading and math when compared to the 2013 results. On the other hand, North Carolina's 4th-graders posted a four-point increase in reading, while dropping a point in math.

    You may have read or heard these results when they first came out. You may have read or heard Democrats making hay of the 8th-grade declines and Republicans taking credit for the 4th-grade reading gain.

    But did you hear this? None of North Carolina's trends, up or down, was statistically distinguishable from the performance track of the nation as a whole. When you are talking about one, two, or even three points of difference on scales in the hundreds of points, what may at first glance seem notable can turn out to be nothing more than statistical noise.

    Every two years, NAEP officials release their reading and math scores accompanied by graphs, tables, and appendices to explain this critical issue of statistical significance. Every two years, the issue gets downplayed or ignored. It's like reporting pre-election polls without including the margin of sampling error.

    As it turns out, only a handful of states had statistically significant NAEP gains (or losses) relative to the national average in any category from 2013 to 2015. Among 4th-graders, the only places to post such gains in both reading and math were Louisiana, Mississippi, and the District of Columbia. Among 8th-graders, the only place where that happened was Arizona. If we broaden the time frame out a bit, starting in 2011 rather than 2013, the list of top-performing states broadens a bit, as well, to include Tennessee, California, Indiana, and Wyoming.

    Of these jurisdictions, Wyoming and D.C. spend more than the national average on their public schools, after adjusting for student enrollment and cost of living, while Indiana, Arizona, Mississippi, Tennessee, and California spend less and Louisiana roughly matches the national average. Indiana, Arizona, Louisiana, D.C., and Tennessee get fairly high scores from conservative education reformers for their policies on standards, teacher quality, and school choice. But California, Mississippi, and Wyoming don't.

    Drawing useful conclusions from test-score data about the successes and failures of education reform requires more than just eyeballing numbers and writing copy for attack ads. Scholars in America and around the world have been producing valid work on this subject for decades now. In general, their findings suggest that in places where standards are high, tests are independent and meaningful, teachers are hired and retained on the basis of real qualifications and ability, and parents have more choices about where to send their children to school, academic performance tends to be higher, even after adjusting for student and family characteristics. By comparison, variables such as per-pupil expenditures, average teacher salaries, and average class sizes do not exhibit consistent relationships with test scores, graduation rates, or other outcome measures.

    For North Carolina, the real news from the 2015 NAEP was that we essentially followed the national trend - unfortunately so, in the case of 8th-grade reading and math.