The Human Rights Campaign is spending millions of dollars on grassroots efforts in Arkansas over the next three years. The goal is to make Arkansans more open to their political agenda–an agenda which includes things like same-sex marriage and expanding “transgender” privileges. The group released a survey this week claiming there is rampant discrimination against homosexuals living in Arkansas.
The trouble is the survey is not reliable. Here is why:
The survey was not conducted at random. Surveyors recruited participants.
“HRC commissioned Anzalone Liszt Grove Research to conduct the online survey of its membership and the LGBT communities across Arkansas. 979 respondents participated in the online survey during the course of six weeks from February 6 to March 20, 2014. Survey participants, which represent various groups within the LGBT community, were recruited through email, social media, and online ads.”
That quote comes directly from the Human Rights Campaign fact sheet on the survey. The questionnaire was completed online by people recruited to fill out the survey–and the fact sheet notes many of those completing the survey were members of the Human Rights Campaign.
This makes the survey fatally flawed–even if the Human Rights Campaign did everything it could to keep the rest of the questionnaire process as neutral as possible. I’m not saying the Human Rights Campaign tried to tell participants how to answer the survey questions. I’m saying the survey does not sound like it was conducted with the same scientific precision Americans have come to expect from popular opinion polling.
No one would take seriously a survey conducted by a conservative organization like Focus on the Family if it was revealed the survey participants were deliberately recruited, especially if the survey participants had some notion ahead of time how the survey might be used.
We saw this very same problem with a recent study purporting to show children of same-sex couples fair better than children of heterosexual couples; in that case, surveyors used advertisements in magazines and at events popular among gay people to recruit participants. Professor Mark Regnerus of the University of Texas at Austin had this to say about the survey technique:
“I don’t know if there’s any other way to say this than to suggest that…this is not the way to build a sense of average same-sex households with children. To compare the results from such an unusual sample with that of a population-based sample of everyone else is just suspect science. And I may be putting that too mildly.”
When it comes to statistics and polling, non-random survey samples are often suspect. If participants are recruited or invited to participate in the survey (through, say, an email or an advertisement), it may indicate they feel they have a vested interest in the study’s results and may impact the kinds of answers they give.
When well known pollsters like Gallup or Rasmussen want to gauge what Americans are feeling or experiencing, they survey people at random. They do not usually invite people to participate in the survey, because that has the potential to impact the survey’s accuracy.
If the goal of this survey was to determine what kinds of opportunities and obstacles homosexuals face in Arkansas (relative to other Arkansans), the most accurate way to do that would be to poll a random sample of Arkansans–not just gay Arkansans–asking them their sexual orientation along with different questions about workplace experiences; their faith; family situation; whether or not they feel they have ever been discriminated against (and the cause of the discrimination); and so on. A study to this effect would follow the tried and true guidelines for unbiased polling.
Of course, a study of that magnitude sounds daunting, but the Human Rights Campaign is spending $1 million per year on its efforts in Arkansas, right now. Surely they can afford to conduct a proper poll, can’t they?
Luke