Vertigo / Group

Previous
Next

Numbers Don’t Lie; Researchers Do.

We have all heard the expression that numbers lie, especially when being applied to statistics. I beg to differ. The quality of the statistics is only as good as the researcher collecting and processing the data. The key to accurate statistics is asking the right questions and then applying proper correlation techniques.

Case in point. I recently read an article on the Wall Street Journal (WSJ) website about a non-scientific survey pertaining to the iPhone finally coming to the Verizon network. Now, reading the survey at face value, it showed that over 70% of users are willing to switch networks and feel the addition of Verizon is a good thing, or so implied by the WSJ.

Here in lies the problem; what specifically is being asked? The WSJ is asking two questions in one. In short, they are asking participants if they are willing to switch networks—from AT&T to Verizon—and are they going to be purchasing a new phone based on the aforementioned information? Number 1, the multipronged question should’ve been asked independently, and secondly, there is little correlation between the questions.

Based on the population size of the survey—roughly 10,000 participants—there is a lot of room for error. Meaning, participants answering the single question, but only responding to the second part affect the overall ratio because of a small population size. So, if most users are simply interested in purchasing a new phone, it also implies that users will be switching from AT&T to Verizon.

As noted above, trusted correlation cannot be attained through two questions. Additional questions needed to be asked; such as contract termination, consideration of contract termination, maximum contract termination fee the participant is willing to pay, satisfaction level with AT&T, participants current carrier, etc…

The critical aspect of simply viewing surveys and polls is to ask how the survey was constructed? What questions were asked? Survey population size. Survey demographics and so forth. The numbers are always correct; it is the researcher or analyst that fucks them up. Sometimes purposefully.

Be Sociable, Share!