Los Angelenos think highly of their hamburgers. Not so much their pizza. According to resident responses, L.A. hamburgers rank fifth best of 35 U.S. cities based on a report by Travelandleisure.com. But L.A.’s pizza ranks 28th, seven spots from dead last. It is no surprise that L.A. residents are less proud of their pizza than are residents of Chicago, New York, and even San Francisco. But do Los Angelenos really believe their pizza is worse than pizza served in Anchorage (6), Savannah (7), Nashville (18), Kansas City (20), and Santa Fe (27)? As a long-time Los Angeles resident and pizza lover, I find this result laughable.
Surveys are used everywhere: political contests, marketing studies, advertising gimmicks, and even as support for expert testimony in high-stakes litigation. While surveys can be performed properly and deliver reliable results, it is surprising just how many fail on both counts. Travelandleisure.com’s “America’s Favorite Cities” survey is no exception.
It not surprising to find that a survey intended solely as a marketing ploy to drive traffic to a travel company’s website is flaw-filled. In fact, for marketing purposes alone, Travelandleisure.com’s survey is quite brilliant. The website is able to attract vast amounts of free press as newspapers and blogs pick up on the survey’s results and report mildly interesting findings that are relevant to their respective readership. Meanwhile, nobody really takes the results all that seriously or gives a second thought to the underlying methodology. So, kudos to Travelandleisure.com for accomplishing its objective. To that I say, “well done.”
However, it is quite surprising when the same poorly considered methodologies that Travelandleisure.com draws upon show up in high-stakes expert testimony work. While no one thinks much about underlying methodologies for marketing gimmicks, survey results in litigation are often highly contested by the opposing party and must be defensible. Obviously, the court and jury expect the designated expert witness to submit as evidence work that is reliable and performed to the appropriate standard of care.
Nonetheless, many expert witnesses conduct and present surveys in a manner that is consistent with Travelandleisure.com’s gimmicky marketing ploy. Litigators and other legal professionals should be aware of the weakness that their designated experts might be submitting as evidence. Your adversarial party will expose these weaknesses by identifying for judge and jury absurd results and/or ill-conceived methodologies.
A series of posts will use Travelandleisure.com’s survey as a case study to demonstrate how your expert’s survey results may be exposed.
