October 04, 2006

On polls and polling data

I felt compelled to write about the recent polls of Cuban-Americans that have been in the news. Val alluded to them in an earlier post. One of my major responsibilities at work is to design, implement and interpret market research. Market research is a fancy way of saying surveys, surveys that are very similar in structure and methodology to legitimate political polls.

First of all I want to say that just because a certain person finances a poll does not automatically mean that the poll should be suspect. For example Republicans may conduct a poll of likely voters to determine which issues to hammer on during an election. It serves them no purpose to skew the results because they need accurate information to make informed choices. Each poll should be looked at independently for validity. The reputation of the pollster and the design of the questions is an important factor in determining whether a poll is valid measure of public opinion.

Another thing to look at is the criteria for recruiting respondents to the poll. Sometimes a pollster only cares about a segment of the population. Sometimes they want a cross section of the entire population. One common trap is to extrapolate the results from a segment (for example, likely voters) to the whole population or vice versa.

The problem with these polls is often not the polls themselves but the interpretation of the results. That's why it's great when we get to see the questions verbatim and the also the demographic data about the respondents.

I saw Sergio Bendixen, the pollster recently conducteed a survey of 600 Cuban-Americans in Miami-Dade and Broward counties, on A Mano Limpia, the show hosted by Dominican-born journalist Oscar Haza.

Bendixen is a very articulate and thoughtful man and I think his conclusions are reasonable. Some conclusions drawn by others (the media, the sponsors of the study, and detractors) are less reasonable. Here's what I mean:

Q: Do you approve or disapprove of the way George W. Bush has managed the situation in Cuba since Fidel Castro handed over the power to his brother, Raul Castro?

A: Approve (51%), disapprove (28%), Don't know/no answer (21%)


Opponents of George Bush (like Joe Garcia whose organization funded the poll) claims this is a sign of eroding support for the president's policies toward Cuba. His assumption is that those who disagree with the President's handling of the situation think the same way that he does, that the our policy is too harsh and unbending. But the question does not ask about the root of the disapproval. It is not unreasonable to believe that some of these people think the President is being to soft at a critical time in Cuban history with the attempted hand-off of power. Another factor to consider is the large number of people that either didn't know how to respond to the question or didn't want to answer it. 21% is abnormally high for a survey. That may signal some ambivalence to the president's handling of the situation among the respondents. What we know for sure is that 72% (a solid majority) don't disaprove of the president's policies.

All in all, I think the results of the Bendixen poll are very favorable in terms of portraying South Florida's Cuban-American community in a positive light.

Q: If a new Cuban government shows an interest in a gradual improvement of relations with the exile community and with the United States, do you think the U.S. government should negotiate with the new Cuban government?

A: Should Negotiate (72%), Should not (20%), DK/NA (8%)

This question is open ended enough to allow the person being asked to envision a scenario under which a negotiation could take place. Pretty much everyone sees a scenario where that can happen. If fidel and raul were to leave Cuba tomorrow, for example, what would be the point of not negotiating with the successor govenment? I think most people are optimistic about a post fidel Cuba (even with Raul) because they understand intuitively that the obstacle to re-establishing normal relations with Cuba is Fidel not the US. That even raul has a greater possibility of doing that than fidel.

This point is emphasized in the responses to another question that Bendixen says show a cautious optimism about Cuba with Fidel out of the picture.

Q: Do you think there is a major probability or a minor probability that there will be a major a transition towards democracy in Cuba now that Fidel Castro has handed over power to his brother Raul Castro?

A: Major probability (55%), Minor probability (30%), DK/NA (15%)

Also telling is the amount of people that say they won't return to Cuba permanently.

Q: If freedom and democracy were re-established in Cuba, would you return to Cuba to live or would you stay and live in the United States?

A: Stay in the US (80%), return to Cuba (13%), DK/NA (7%)

Bendixen did a good job of explaining that the roots in the US for many Cubans are now permanent. Although he does believe that many will travel back and forth while keeping their American citizenship and residency.

Along the same lines and perhaps the most important revelation is about property claims.

Q: Do you believe that residential and dwelling properties in Cuba should belong to those who have title to that predating Fidel Castro’s Revolution?

A: To those who have atitle to that property predating FidelCastro's Revolution (20%), To those that live in them now (67% ), DK/NA (13%)

Most Cuban living in the US understand that it would be unfair to pull Cubans out of their houses. A big blow to the Cuban propaganda machine that is always trying to trick Cubans into thinking they will lose their crumbling homes if the "exiles come back." With most saying they won't come back and most saying that Cubans should be secure in their homes that argument is revealed to be hollow.

I just want to close by saying that we can learn a lot from these polls if we scrutinize them well and are open the possibilities of what the responses could mean.

Posted by Henry Louis Gomez at October 4, 2006 12:56 PM



Trackback Pings

TrackBack URL for this entry:
http://www.babalublog.com/cgi-bin/mt/hut.cgi/4059

Comments

I didn’t watch Oscar Haza’s program so please excuse my ignorance. I have some questions about this poll:

How was Bindixen able to target only Cuban-American households in Miami-Dade and Broward counties?

Did they use the membership list of the CANF (Cuban-American National Foundation) for this poll? If no, then what source did they use?

Were these questions addressed to those BORN in Cuba or to their descendants (first generation Americans)?

Though Cuban-Americans comprise a large ethnic group in both Miami-Dade and Broward counties, when you total ALL the other Hispanics living in both Miami-Dade and Broward counties Cuban-Americans are the minority.

Posted by: Firefly [TypeKey Profile Page] at October 5, 2006 10:20 AM

Henry:
Good job of lending your interpretation of the poll results. While I think you are true to the final statement of your post, that "we can learn a lot from these polls if we scrutinize them well and are open the possibilities of what the responses could mean", let me offer some good-natured counterpoints to some of your statements, if for no other reason than to spark some discussion (though I do think Firefly poses some interesting questions which I hope you can shed some light on).
In your analysis of the Bush "referendum" question, you correctly point out that the form of the question itself could have been subjectively interpreted by the respondent, making interpretation of the polled results difficult. It is true that at best, disapproval of the President's performance still left open the question of the root of the disapproval, and that such disapproval could be cast on either side of the status quo. I would also agree that the DNK's were rather high for an "up-or-down" question. What I would counter, however, is your assertion that, "What we know for sure is that 72% (a solid majority) don't disaprove (sic) of the president's policies." You seem to adopt the very spin you attribute to Joe (the wholesale adoption of the DNK responses) to shore up your preferred conclusion. That doesn't seem right, does it? What we know "for sure" is that according to this poll, 51% (a simple majority), and perhaps more, don't disapprove of the status quo. Even Joe's conclusion can't be properly assessed without a baseline.
I have no problem with the "gradual improvement" question, inasmuch as it is an open-ended question and therefore prone to positive responses which may outstrip the intent of the question, because I don't believe that to be the case here, in the context of your analysis of the results. Your conclusion that "most people are optimistic about a post fidel Cuba (even with Raul) because they understand intuitively that the obstacle to re-establishing normal relations with Cuba is Fidel not the US," is perhaps at the limit of conclusions that can be drawn from the responses to this particular question.
The Fidel/Raul "probability" question does give me pause, however. Once again, I think your analysis confirms the intent of the question (would you agree it shouldn't be this way?), in that you properly focus your conclusion as between the brothers castro and not, as the form of the question would suggest, that the transfer of power to raul made a transition towards democracy "a major probability." Something tells me something may have been lost in translation on this one...
I would suggest that the "property" question is murky, in that it mixes issues that are not mutually inclusive. The issues of habitation, which I think benefit from the humanistic conclusions you adopt, and the issues of restitution for illegal expropriation can co-exist, if it comes down to it. And-- as fair-minded as we Cubans (an certain lower life-forms) can be-- with money, it always does.
Anyway, just two or three cents-worth... and no more than that.

Posted by: nonee moose [TypeKey Profile Page] at October 10, 2006 02:23 PM

Nonee moose,

Good to see you over here. Regardless of what's said this place is full of serious and usually respectful disagreement. I answered Firefly's questions via email because we have a running dialogue going.

You bring up valid points about the Bush question. But would it be untrue to say more people voted against Clinton in 1992 than voted for him? Remember that Ross Perot took a significant percentage of the vote. The result was that Clinton (if memory serves) only got 41%. I could re-phrase the interpretation of the responses to say only 28% of those asked disapprove of Bush's handling.

In polling sometimes you have to join respondent groups together. I'll give an example. I show you a commercial and ask you if (1) you liked it a lot (2) you liked it a little (3) you neither liked it or disliked it (4) you disliked it a little or (5) you disliked it a lot. Usually people tend to congregate around the middle unless the commercial (or other question) is really polarizing. We don't necessarily know why the people that disliked it disliked it. Same for the ones that like it. You need to ask follow up questions. But each question makes the survey longer and that means it's more expensive and harder to field. Anyway we might add the ones that liked a little and the ones that liked a lot and generically say that this is the number of people that liked the commercial.

One of the things that I want to point out is presidential approval ratings. We are told by the media how low the president's approval ratings are. But what is rarely talked about is why? Certainly there are a lot of people out there that view themselves as ideologically opposed to the President. For them nothing he will do will meet their approval. These are liberal partisans. Then there are conservative partisans. Some of them have been turned off by the president's policies but not for the same reasons as the liberal partisans. To them he's not living up to the conservative billing. But faced with a choice between Bush and say Kerry in an election they will almost without fail vote for Bush. In our deeply and evenly divided country this is important to understand. In the Bendixen survey there were a significant number of people that maybe aren't 100% happy with Bush's handling of the situation but they weren't unhappy enough to say they didn't approve.

As for the Fide/Raul question I think the way it's phrased speaks for itself. The respondent is supposed to predicate the response on the fact that "Fidel Castro has handed over power to his brother Raul Castro" Obviously if we had no prior knowldege of the situation we might have a different interpretation but the interpreter should know some of the history. Like the other question (and almost any other question) the respondent can interpret this in many ways. Here's a few examples:

A major probability because Raul is more pragmatic and less dogmatic than Fidel

A major possibility because anything is better than Fidel

A major possibility because Raul's own general's aren't going to let him rule.

A major possibility because Raul is an alcoholic that will wilt when faced with major decisions.

My only point was that while most Cubans in SoFla would probably like to see a bolt of lightning strike both brothers at once that there is cautious optimism that any change (even to Raul) has to be for the better. It's an idictment of fidel that people can't imagine Raul being worse even though he's been sold as such by Fidel himself.

Thanks for reading.

Posted by: conductor [TypeKey Profile Page] at October 10, 2006 02:56 PM

Henry:
I must confess, I have heard some of the things said about this place, as you well know. I was brought up not to believe everything you read or hear, and that lesson has been reinforced the hard way over the years. So until I get banned or censored for my occasional strenuous objection/disagreement, you have the benefit of any doubts I may have. Mostly, I like to argue for kicks...

You asked a question, " But would it be untrue to say more people voted against Clinton in 1992 than voted for him?" My answer is this: impossible to ascertain, given the facts at the time. I believe a more accurate statement would be simply that more people did not vote for Clinton than did. Allow me to explain why.

The 1992 Presidential election was much like so many others in our history, in that a sitting president was seeking re-election. I believe therefore that the only "referendum", in its simplest and most precise sense, that was in play was on the current administration. I recognize this is overly simplistic (clearly, all the candidates had some public history which was subject to scrutiny), but given the job description and position all were vying for, only one man had the exact experience which could be accurately assessed. Based on that, the only votes which could most accurately be characterized as "against" any candidate would have been those cast "for" someone other than the man already holding the job. Again, I realize the difficulty (impossibility) of objectifying something so subjective as a vote. My point is only that reciprocal conclusions cannot be drawn in your example.

I agree with you that sometimes you can combine groups of respondents together and draw generally accurate conclusions. This practice is especially useful in the science/art of spinning. The particular example you used to illustrate your point is especially useful to my point, which is this: no matter the multiple of choices you use (5 in your example), there are only ever 3 categories. One is, in absolute terms, the polar opposite of the other, and the last is neutral and cannot be claimed in support by either of the other two. At least not with intellectually honest precision.

Your discussion on presidential approval ratings further supports this point. Again you recognize that "disapproval" can theoretically fall on either side of the status quo for the reasons you stated. But that fact alone cannot serve as a basis for attributing any characteristics to that "third column" beyond its facial neutrality. Follow-up and further probing or narrowing of questions would certainly allow a greater basis for attribution, certainly. But without that, "disapproval" is just that-- "disapproval". Which is why "undecided" is such an X-factor in election polling. You have to first calculate the probability of an undecided voting at all, THEN the probability of that vote being one or another. Your assertion that, "... faced with a choice between Bush and say Kerry in an election they will almost without fail vote for Bush..," assumes they would vote at all. You know as well as I do that, without fail, even conservatives stay home sometimes. I will concede all the known data on voting patterns of certain demographics. In an absolute sense, it is anything but.

With respect to your interpretation of the fidel/raul (btw, I don't know who started the subtle diss of not capitalizing... nice touch, and it has caught on elsewhere...), my point was merely that the question seems to assume a "major probability" of democracy on the basis of the transfer of power. castroLite's pragmatism aside, calling anything a "major probability" based on no more than speculative comparisons of the brothers just brings out the cynic in me. I guess I just have a bias against major probabilities, after so many weather forecasts...

And for the record, may the bolt of lightning be so big that it fries the early-warning system (poor Cheo with a wire-hanger) in La Habana...

Posted by: nonee moose [TypeKey Profile Page] at October 11, 2006 03:31 PM

I agree with your analysis. But some times it's 6 of one half dozen of the other. The purpose of my post was to get people to understand that depending on the reference point two people can have different interpretations of the same research.

As for the subtle diss against fidel (lowercase) Val started that a while back. And to be honest with you I think its kind of appropriate. The man doesn't really deserve any of the ordinary respect that you would offer a gentleman.

As for the fidel/raul question it's not a gauge of actual probabilities but of feelings of optimism/pessimism. I think everyone has fidel fatigue and the news of his illness came out of the blue and was very dramatic and gave people license to dream about a Cuba without fidel. And I don't think there is anything irrational about that. The worst despots are almost singular in their personality. In other words, there was probably only one Hitler in Germany. Any other man that could have risen to power may have been terrible but it's hard to imagine a man as terrible as Hitler himself. And so it is with Fidel. Most Cubans (in my opinion) believe they have seen the worst and there's only one place to go: up.

Posted by: conductor [TypeKey Profile Page] at October 11, 2006 03:58 PM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?