Measuring Value Added is Essential | Eastern North Carolina Now

    Publisher's note: This article appeared on John Hood's daily column in the Carolina Journal, which, because of Author / Publisher Hood, is linked to the John Locke Foundation.

    What difference does it make?

    No, I've not become jaded, defeatist, or desperate. I'm naturally optimistic. While perfection is impossible in our world, things can be better than they are. Through purposeful action, all of us can help make that happen.

    As a conservative, I don't assume such purposeful action must be organized or funded by government. In fact, government intervention, because it is forceful and subject to perverse incentives, often fails to accomplish its intended goals and can even make a problem worse by misdiagnosing its causes, provoking a backlash, or otherwise producing unintended consequences.

    To favor voluntary solutions and incremental gains over coercion and central planning is not to favor the status quo or dismiss the idea of progress. In fact, the past two centuries have featured the greatest leap forward in human well-being in the recorded history of our species.

    The Industrial Revolution, the expansion of freedom, the birth of representative government, and the technological marvels of the modern age have dramatically increased average living standards and longevity. We are far wealthier and healthier than previous generations were.

    Whether one considers market innovations, social reforms, medical discoveries, or public policies, a prerequisite for progress is figuring out how to avoid false positives and false negatives. That is, if your business introduces a new product and you make more money the following year, was it the new product that did the trick or the extra effort you put into marketing and customer service? If a one doctor has lots of healthy patients and another doctor has lots of patients who suffer and die agonizing deaths, is the second doctor less skilled or does he simply attract or seek out more difficult cases? If a state legislature passes a new safety law and then accident rates drop, did the law cause the improvement or were accident rates likely to drop for other reasons, anyway?

    If you conclude incorrectly that something you did had a positive effect, you can end up wasting lots of time and resources doing something that doesn't really produce a net benefit. And if you conclude incorrectly that something you did had no positive effect -the problem got worse, but actually it would have been even worse than that without your intervention - then you may also misallocate resources to a lower-value use.

    One obvious answer is to devise careful tests with experimental and control groups. With human beings, however, this isn't always possible, popular, or ethical. So another idea is to employ statistical methods to control for other factors and then attempt to detect the "value added" by a particular program or person.

    There is always resistance. Here in North Carolina, for example, some teachers have strongly objected to the use of value-added assessments for evaluating schools, instructional practices, and teachers themselves. They argue that value-added testing isn't free from flaws or biases, which is certainly true.

    But that's not a sufficient objection. Techniques for measuring value added and using the results to make management decisions are increasingly common through all professions and industries even though perfection is clearly impossible. That's because the alternatives are worse. They generate even more false positives and false negatives.

    In the latest issue of the Quarterly Journal of Economics, a team of researchers from the Massachusetts Institute of Technology and the University of California at Berkeley used education data from Boston to test the validity of value-added assessments of the schools there. They did find some bias. Nevertheless, the scholars concluded, the proper use by policymakers of the available value-added assessments "are likely to generate substantial achievement gains."

    Common sense tells us that principals and teachers, like all other professionals, differ in their abilities, diligence, and effectiveness. We want to retain and productively deploy the highest-performers, help the middling performers get better, and suggest that the lowest-performers pursue other careers. Even as models get refined, some educators may get misclassified. But flying by imperfect instruments is still better than flying by the seat of your pants.