A recently published article on “Weight Bias in Graduate School Admissions” (gated link here) inadvertently highlights the dangers of drawing global conclusions based on inappropriate samples. Based on responses from 97 applicants to one university’s psychology department, the study asserts that graduate applicants with a higher Body Mass Index (BMI) tend to receive significantly fewer post-interview offers of admission. This small and specifically defined sample is probably unrepresentative of the entire population of graduate school applicants for the following reasons:
- The sample consists of applicants to only one graduate institution (most likely Bowling Green State University in Ohio) described as “a large Midwestern university.” Applicants differ dramatically based on the schools to why they are applying. For example, prestigious universities, as well as those known for excellence in a particular discipline, obviously attract much higher caliber applicants (i.e. higher GPAs, GRE scores, better letters of recommendation) relative to lower ranked schools. Therefore, by focusing only on a lower tier university like Bowling Green State, the sample almost certainly excludes important segments (i.e. the highest academic achievers) of the population of graduate applicants.
- Only applicants to particular psychology department are included within the sample. Academic departments vary tremendously both in terms of demographics and academic achievement. For example, relative to English or psychology graduate programs, engineering typically attracts far more males and students with more technical and mathematical backgrounds. Limiting the sample to only one department again ignores significant segments of the population under study.
- Like many surveys, this study suffers from “non-response bias” because those who chose to respond to the survey were likely different in important ways relative to those who failed to respond. In this case, non-response bias is likely to be highly problematic. Of the 274 applicants who were invited to participate in the survey, one third (or 97) chose to respond. One explanation is that applicants were unwilling to report sensitive information about their weight. Additionally, because the survey was conducted after admissions decisions had been announced, it is likely that some rejected applicants might be unwilling to respond to the university’s requests. This cause of non-response, which is systematically related to applicant quality, means that the sample obtained almost certainly overestimated the quality of applicants even with respect to this university alone.
When a sample is both small and unrepresentative, the conclusions drawn from it should be scrutinized. Although the claim that overweight applicants are disadvantaged in the graduate admissions process sounds plausible, this article does not succeed in demonstrating this result scientifically.