Results from a Maui Now poll ricocheted around social media Monday within minutes of the erstwhile news website’s publication of its “survey,” which looked like bad news indeed for Mayor Michael Victorino. The “survey” (and I’ll get to why I keep putting quotes around that word in a minute) of Maui residents was conducted over a seven-day period earlier this month and received 1,935 responses, according to Maui Now. It reported that 81% of survey participants disapproved of Victorino’s handling of the COVID-19 pandemic. Gov. David Ige fared even worse, with 85% disapproving of his pandemic decisions. In other survey results, which were presented in all kinds of official looking colorful graphs and pie charts: 67% of respondents were “very concerned” about Maui’s economy. Roughly half were concerned about COVID, as opposed to 54% who were not. Finally, 85% of respondents disapproved of Hawaii’s vaccine mandate policies.
However, there’s a major problem with the survey’s rather incendiary findings, says Colin Moore, the director of the Public Policy Center at the University of Hawaii. “It’s junk. This is just a bunch of meaningless garbage, similar to the polls that proliferate on Facebook.”
Moore, who has helped formulate many respected surveys, such as the Center’s “Managing Tourism in Hawaii” report published in June, faulted Maui Now’s effort on almost all levels.
“These 1,900 people just responded to some sort of solicitation. That may sound impressive, but those respondents are highly unlikely to be representative of the population at large,” he explained. “The entire point of a poll is to use a sample of the population to represent the entire population. But to achieve that, you need a random sample.”
Emphasis on “random.”
When it comes to those who participated in the Maui Now survey, Moore explained, “The social science word is ‘selection bias.’ Either it’s biased just to the readers of Maui Now, or it can be manipulated because people can be encouraged to participate in this poll who have a certain view.”
Although Maui Now proudly proclaimed that it checked all respondent’s email addresses so that there were no duplications, what would prevent a highly paid lobbyist from firing off an email to clients asking them to respond to the survey during the week-long response time? Or the head of an anti-vaccine organization? Nothing, says Moore.
“Any legitimate poll would publish the full research instrument with a detailed description of exactly how respondents were recruited. In this survey, respondents just clicked on a link. This is like what the magazines do. They ask, ‘What’s your favorite restaurant?’ Then anyone can click in and support.”
Real Polls are Expensive
The danger, Moore says, is that “People see this and they think it’s a real poll. We know that polls can have the ability to shape opinion. This survey is irresponsible and worthless because it doesn’t do anything more than show what the 1,900 people who clicked on the poll thought.”
Additionally, the size of the response bothered Moore. “Even the highest quality national polls only range from 1,000 to 1,500 carefully selected respondents. To conduct a legitimate survey on Maui of 1,900 people would cost about $50,000.”
I reached out to Maui Now editor Wendy Osher and her boss, Pacific Media Group Chief Operating Officer Jack Dugan, for comment on Moore’s criticisms and received no response.
This latest stunt just increases my heartburn about the Maui Now website in general. With some of the best analytics and the widest reach on the island, Maui Now could be a strong force in furthering strong reportage on Maui. But it doesn’t do that. Untouched press releases constitute more than 90% of everything listed on the news site. And they’re never identified as press releases. Now I’ve written many, many stories based on such items, but the trick is “adding value.” In other words, using the news release as a stepping stone to further reporting and interviews to more fully round out what is usually a one-sided presentation of a story or issue.
Maui Now just prints the release as a news story. For example, the recent clean-up on Amala Place produced a flurry of news releases from the Victorino administration. Each one was run verbatim by Maui Now. Last week, homeless advocates issued a release of their own. Rather than print that one verbatim, editor Osher wrote one of her rare stories that cherry-picked info from that release and interspersed it with comments from the county. Maui Now’s policies aren’t fairly applied and, in the majority of instances, favoritism seems to be granted to the organizations who have the money and staff to write their own news.
With this “survey,” Maui Now is venturing into even more dangerous territory, one that could have unsettling repercussions if the website continues its shoddy survey practices during what will be an undeniably heated election cycle in 2022.
They put a survey on their website, and posted the results. Not sure why you wasted your time making an issue out of this. I re-read their results article, and don’t see any mention to the contrary. Sad to watch Maui’s limited local news sources bring each other down. I enjoy the articles on your site, as I do the Maui News, Maui Now and Maui Times. All have different stories that I might’ve missed if I just followed one.
Matt, Thank you for your comment and for being such a thorough reader of local Maui news. The only way to stay truly informed is to read as widely as you do. My response is this: How would you feel if one of your news sites printed a story that was absolutely not true? In the case of the Maui Now survey, that’s exactly what they did. They presented a “survey,” compete with charts, graphs and degrees of accuracy, as if it were a real poll. As Colin Moore explained, it wasn’t that at all, for all the reasons he outlined. News organizations that conduct political polls like this pay substantial fees to organizations specializing in designing and conducting a verified, accurate survey of a random population group. That’s to maintain credibility and trust. As we saw in the last presidential elections, even those carefully organized polls can prove difficult. However, for a news organization to not follow ANY of these guidelines and suggest that it has conducted a “real” survey is just as bad as a reporter who thinks it’s okay to make up quotes. The Honolulu Star Advertiser has a “Question of the Day” feature that asks readers to vote on various questions. Had Maui Now presented its questions that way, there would have been no need to write a story.
Good goin’, Ms. Rybak. I’m a long ways from this action, but flat-out bullshit metrics presented as fact annoys the bejeezus out of me. (Can I say “bullshit” and “bejeezus” in your comment section?) Anyone … anyone … purporting to be trading in information in a professional manner has an absolute obligation to basic due diligence. As in caring and demonstrating that what they’re talking about is accurate and not just more bullshit clickbait. (Apologies again.)