How many times have you heard someone say “I have never been asked to take part in a survey? How can it be true if I was not asked?” Not everyone is asked but everyone has to have an equal chance of being asked. To have this happen one has to look at how a pollster selects his respondents.

What did they say?

HHB: “…using a random sample…drawn from HHB and Associates’ national representative adult panel…”

SBS: “…sampled from listed residential telephone numbers and valid mobile phone numbers by the random digit dialing method…the demographics of those polled approximate the ethnic profile and age distribution of the national population…”

These are two very different ways to draw a sample. This could have an effect on the outcome of the surveys and whether or not they are representative of the national population.

HHB is using members of their panel and SBS is using a telephone approach to recruit. What does this mean? In the absence of details about the final sample, let us look at the way they reported their selection approach.

A survey panel is a sample of respondents who have agreed to take part in multiple surveys over time. The big advantage of this approach is that one can track changes in attitudes and behaviour of the same individual over time. This is advantageous in that you can build a good knowledge of the participants but the weakness is that repeated questioning may lead to different answers from those obtained from a ‘fresh’ sample. If the questions are the same, the person may feel pressured to give a set answer to appear to be consistent, especially if they believe this is desirable. HBB also does not describe how they get the random sample from their panel nor do they report on any changes in opinion (remember they did the last election polls) from these members of their panel so we are at a loss.

The method used by SBS is a combination of telephone surveys – landlines and mobile lines. This is important to improve coverage – no longer can we reach the national population only by landlines, some persons now have no landline. Young people do not generally have their own landline and only use their mobile, therefore, they have to be reached via the mobile. However, the screening for persons to be used in a mobile phone survey needs to be more diligent since one does not have the luxury of validating the residence details nor any other demographic easily. The landline approach also enables you to select from those at home (more than one person) but the mobile is individual and this affects the way your sample is created. The caller identification feature, which mobile phones all have, also can change the response rate for this group. It is therefore important to understand what was the proportion for each group of respondents, how the data was combined and how was it weighted before being used.

There is a US study that demonstrates that landline users have supported one party over another – a factor that may be age/lifestyle related. Is this true of us here? SBS does not provide critical data about their blending of the data nor about the proportions of each sub-sample. So we are left at a loss.