In a previous post, Eric discussed a Brookings Institution policy brief that argued, among other things, that college education is a poor investment for some students. This assertion is based in large part on the negative return on investment (ROI) reported by Payscale.com for certain non-selective colleges. The problem with this sort of analysis, however, is one of confusing correlation with causation. Sure, Harvard grads on average make more money than alums of Savannah College of Art and Design (one of the schools with negative ROI in 2012), but are the students who attend these very different schools identical in all relevant aspects except the school’s name printed on their diplomas? Clearly, the answer is no. Harvard students on average possess a high level of intelligence and motivation that should result in generally higher earnings with or without an Ivy League education.
This problem, which is known among economists and statisticians as selection bias, is a thorny one that even the authors of the Brookings Institution’s brief recognize:
However, correlation is not causation, and getting at the true causal effect of education on earnings is not so easy. The main problem is one of selection: if the smartest, most motivated people are both more likely to go to college and more likely to be financially successful, then the observed difference in earnings by years of education doesn’t measure the true effect of college.
Unfortunately, the authors make this disclaimer, but proceed to discuss earnings correlations and offer policy recommendations as if the selection bias had somehow been overcome.
Many research questions posed by economists must wrestle with potential selection effects. For example, a researcher wanting to show that high government debt causes sluggish economic growth must establish that those countries choosing to rely heavily on debt are equivalent in all other important dimensions to those that don’t. Because of the pervasive nature of selection effects, econometricians have developed several new methods for identifying causal effects.
One such method is called Regression Discontinuity Design (RDD). This method takes advantage of cutoffs (e.g. GPA eligibility) by examining subjects immediately above and below the threshold with the expectation that these two groups are roughly equivalent in all respects except their receipt of the treatment of interest. If the treatment has an effect, the researcher would expect to see a large jump (i.e. “discontinuity”) in the outcome observed in subjects immediately above the threshold relative to those immediately below it.
A recent paper by economist Seth Zimmerman employs this approach by examining earnings from students whose GPAs were barely above and barely below the GPA eligibility threshold for Florida International University. Zimmerman finds that students immediately above the edibility cutoff have significantly higher earnings in later years relative to students immediately below the cutoff. The plot contained in Zimmerman’s paper illustrates the approach and findings nicely in graphical form:
Compared to the Brookings Institution’s brief, Zimmerman’s paper presents findings that are more consistent with the generally accepted belief that college is a worthwhile investment. Notably, Zimmerman finds that even for those marginal students with significantly poorer academic performance, a college degree makes a big difference in future earnings. Of course, Zimmerman only examined one university and that university is reasonably selective and doesn’t have a negative ROI.
In sum, the Brookings Institution’s brief raises some important questions about post-secondary education, but may not be capturing the casual effects of going to college.