A prime example: Immediately after the conclusion of the debates this month between presidential candidates Al Gore and George W. Bush, as well as veep hopefuls Dick Cheney and Joe Lieberman, several networks cut to correspondents or pollsters standing by with voters who were asked for reactions about what they just saw. In most cases, disclaimers were made noting that the results of these mini-surveys weren't scientific, but the amount of time spent on them undercut the concessions. So, too, did the focus on gee-whiz technical tools such as "perception analyzers" -- video-game-like dials that allowed participants in assemblies overseen by political op/MSNBC consultant Frank Luntz to register their pleasure or displeasure about a comment with a flick of the wrist. The colored lines representing their responses that squiggled across Lieberman's face in replays of his October 5 remarks suggested an outtake from A Clockwork Orange -- except in this case, viewers weren't forced to watch by being strapped to a chair with their eyelids pinned back.
Given the mind-numbing content of the debates, that was a surprise. Even the Denver Post couldn't tell them apart: On October 12, the day after the second debate, the paper ran a front-page photo from the first debate. Credit the Post for later acknowledging this gaffe in an article rather than a buried correction.
Other instances of insight-free polling are easy to find; Internet technology has led to an explosion of arithmetically suspect surveys. Admittedly, interactivity stunts can be amusing: ESPN's October 8 broadcast of the Jacksonville-Baltimore football game was highlighted by a real-time poll that showed how many viewers wanted a call to be overruled while a referee was still viewing footage of the disputed play. Yet when it comes to more substantial matters, the possibility that media consumers will be fooled into thinking that something holds water when it's full of holes isn't quite so funny.
Colorado's principal pollsters -- Floyd Ciruli of Denver's Ciruli and Associates and Paul Talmey of Boulder's Talmey-Drake Research and Strategy Incorporated, as well as Fred Bierman, who helps oversee research done for Channel 9 by New Jersey's Survey USA -- are often critical of today's poll mania, even though it's been undeniably good for their businesses. And while they insist that their approaches prevent stat abuse from weakening their work, they occasionally disagree over methodology. Moreover, their revelations about how their material is disseminated imply that news organizations are sometimes more interested in maximizing their polling investment than in putting facts before the public in the timeliest manner.
Ciruli began plying his trade in 1975 and settled in Denver a few years later following the retirement of Congressman Frank Evans, a Pueblo Democrat for whom he'd been working. He initially specialized in consulting and polling for corporations and nonprofits such as the Denver Chamber of Commerce, which hired him to determine if locals wanted a new airport. (Don'tcha wish someone would've asked you?) But in 1987 he inked a contract with Channel 9, which has maintained a relationship with him ever since. Thirteen years later, he still concentrates on the Colorado market, but he's become a nationally known political commentator as well, dispensing wisdom in the Christian Science Monitor and other publications. Thanks to the explosion of the polling profession, he's got plenty of company. "Nationally, we are inundated with polling," he says.
Like Ciruli, Paul Talmey has a background in Democratic Party politics; among other things, he was Roy Romer's assistant campaign manager in the leather-jacketed one's unsuccessful 1966 bid for the Senate. "We drove around the state in a VW bus with speakers on it playing a song to the tune of 'Hey Look Me Over' called 'Hey Look at Romer,'" he recalls between guffaws. "We learned a lot from that race -- like how not to run one." He developed a partnership with Bob Drake in 1980. Today the company continues to work with Democratic candidates in other parts of the country but avoids doing so in Colorado to stave off conflict-of-interest charges. Talmey calls the work his firm started doing for Channel 4 in the early '80s "the first legitimate TV polling done in the state" and notes that its worth was solidified during Denver's 1983 mayoral competition, "when we were showing this guy Federico Peña, whom no one had ever heard of, coming out on top. And guess what happened?"
Talmey has a staff of trained interviewers (around 75 of them), and Ciruli contracts with Ruby Standage Incorporated, a research outfit, but their polling mode is quite similar. For each poll, they carefully line up a representative sample of respondents (500 is a common number) who are then quizzed by staffers armed with queries designed to elicit feedback about both the central issues and the folks themselves. "When we polled about this year's ballot initiatives early on, we also asked, 'Have you heard anything significant about these initiatives?'" Ciruli says. "And even among voters likely to vote, most of them hadn't. I mentioned that in every interview I did to put the poll in perspective."
In addition, Ciruli and Talmey prefer to poll over the span of more than one day, thereby hitting as much of their target sample as possible, rather than engage in the "instant" polling that's become endemic in this age of short attention spans. "Instant polling can have sample problems -- and then there's the way opinions work their way through society," Talmey says. "The polls right after the first debate showed that Gore won, but as it started coming out that some of what he said might not have been true, he dropped like a rock. And that's not something you can see in an instant poll."
The two are equally contemptuous of polling done with the use of recordings -- which just happens to be the way Survey USA operates. The company, which polls for Channel 9 and around fifty other TV stations nationwide, purchases a random sample of phone numbers from companies that specialize in gathering such information, then contacts residents via an automated system. Pre-taped local anchors make the inquiries (Jim Benemann, Adele Arakawa, Ward Lucas and Kim Christiansen pull this duty at Channel 9), with call recipients required to push buttons on touch-tone phones corresponding to their opinions.
The biggest drawback to this technique, Ciruli and Talmey say, is the inability to judge whether someone is being honest, or even to know if he's an adult or a child. But Survey USA's Bierman says his research indicates that his machines bring out the truthfulness in people. He cites Proposition 187, a 1994 California proposal that sought to deny educational access and other benefits to illegal aliens: "All the exit polling done by traditional pollsters had it at 50-50, but we had it at 60-40. In the end, it passed 60.3 percent to 39.7 percent, and when we looked at why, we discovered that with that kind of controversial issue, people were more comfortable telling a recording how they really voted than they were telling a person." Call it the shame factor.
Clearly, Channel 9 has confidence in Survey USA's work: It often kicks off newscasts with its data. That's typical of media outlets, which seem to see polling as a way to buy a front-page headline or lead story. A poll about candidates and ballot measures that Ciruli conducted a few weeks back collected info on many topics at the same time, but the Post, Channel 9 and KOA radio, which shared the survey's cost, dribbled out the findings over five days in coordinated fashion: One day they dealt with abortion, the next day, it was medical marijuana and the gun-show loophole. Talk about getting more bang for your buck.
Nevertheless, these polls at least have a statistical foundation to them, unlike so many that are out there at present. Ciruli grants that bad polling can undermine the credibility of those who do their job the right way.
"When I first started twenty years ago, I spent huge amounts of time explaining how it's possible to take 500 people and get a representative result -- but that's broadly accepted now," he says. "The new phase is skepticism over whether it's being done correctly. That can be a good thing, but it definitely requires everyone in the polling industry to defend their work. And they should."
