Strategies And Experiences Of Telephone Surveys In The World

Socio-demographic differences between mobile and landline phones users

Differences in mobile and landline phone owners involve key socio-demographic characteristics of respondents: age, race / ethnicity, gender, household structure, welfare level, geographic region, and household ownership status (Blumberg & Luke, 2017; Mardian et al., 2010), and this is true for every country, though in varying degrees (Blumberg & Luke, 2009). This is caused both to the difference in general practices of telephone use and to different tariffs for different types of communication (Barr, van Ritten, Steel, & Thackway, 2012; Mohorko et al., 2013; Brick, Flores Cervantes, Lee, & Norman, 2011; Hu, Balluz, Battaglia, & Frankel, 2011). Generally young people as well as working people are over-represented among mobile phone users; underrepresented - people with low education level. Renters are much more likely to use only cell phones, unlike homeowners. This also affects the significant difference in the age of respondents, as homeowners are much older than renters (Vicente et al., 2009). Within a large group of people for whom both types of communication are available there are significant differences between those with whom contact can be more quickly established by mobile communication, and those that are more accessible through landline phone numbers (Keeter, Dimock, Kennedy, Best, & Horrigan, 2008) (Table 01 ).

Specifics of data collection through mobile and landline numbers

The use of mobile communications has several features, the key ones of which include the difficulty of forming the sampling frame, the cost issues of this type of communication, and the problem of cooperation and refusals.

Sample frame

In most countries there are no telephone directories of mobile users, and if they exist they suffer from a wide variety of mobile operators, from the fact that a user can have more than one SIM card, and from the lack of connection between the telephone number and the geographical location of the subscriber. The lack of geo-referencing of mobile numbers makes cell-phone surveys effective for nation-wide research, but causes problems in regional and local studies. The absence of telephone directories forces us to sample mobile telephone numbers using random generation, which leads to the creation of a large amount of non-existent numbers; usually 60% or more (Vicente, Reis, & Santos, 2009). Both problems force researchers to spend significant resources on finding technically accessible numbers and screening potential respondents.

Costs

The charging system for mobile phone services adopts one of two principles: the calling party pays or the receiving party pays. In the United States and some other countries the main principle is "mobile party pays", i.e. he subscriber of a mobile number pays for both incoming and outgoing calls. Surely, the need to pay for a conversation with the interviewer will affect the willingness of the respondent to participate in the survey. Therefore, the research company should take care of how to compensate the costs and encourage potential respondents.

Response Rate

Typically, mobile phone surveys show a lower response rate (Brick, Edwards & Lee, 2007). This may be due to the reluctance to receive calls from unfamiliar numbers, because a mobile phone is a personal device, and the interviewer's call can be regarded as an invasion of private space. Here the respondent's mobile tariff also plays its own role, especially in conditions if the subscriber himself pays for incoming calls. On the other hand, mobile phones allow interviewers to dial at any time of the day, which made such groups of people who had previously dropped out of surveys available for research. The time period for dialing, which used to be mostly evening or weekend time, is increasing. Even holidays, which are characterized by a low response rate for fixed phones, for mobile phones are one of the best periods for making contacts.

With the development and cheapening of cellular communication, the differences between mobile and fixed phones are largely erased, and if they exist, they are rather in favor of mobile communication. But in the recommendations of AAPOR it is said that on many issues of the use of mobile phones in mass surveys it is still not possible to give a definite answer and with full confidence to recommend to use this or that option. For example, RDD samples of cell-phone numbers demonstrate better coverage of the general population than RDD samples of stationary numbers, but it is not clear whether mobile samples must be supplemented with stationary number surveys, and if so, should such dual frame design be screening or overlapping (Analytical statement, 2010).

Sampling and the multiplicity problem.

Coverage bias is one of the main problems in sample surveys. And the most frequently used variant of its solution is dual frame samples that use both fixed and mobile phone numbers (Buskirk & Best, 2012). But the majority of landline phone owners today also have cell phones at their disposal, which makes it more likely that such people (“dual users”) will get into the sample. This problem can be avoided by removing the “dual users” numbers from the list of mobile phone numbers. We'll get a stratified probability sample, where the list of all household fixed phones and the list of mobile-only phone users are represent two separate strata (Analytical statement, 2016).

This screening design, when landline phone owners are excluded from a cell phone survey, was considered successful in the late 2000s. Today, when the cost of cellular communication approaches the landline, and the exclusion of landline phone owners from cell phone surveys leads to the exclusion of a significant number of young people who use mobile phones more actively, a dual frame sample without screening or dual frame overlapping design, when the owners of cell phones are interviewed regardless of whether they have fixed phones or not, is considered to be the most adequate (Figure 2).

But in any of the DFD research options, one of the main issues is the optimal ratio of mobile and stationary telephone shares. One of the answers is the following: the proportion of interviews on mobile phones should correspond to the proportion of the population using only mobile or mainly mobile communication. The rest part of the interviews is conducted through landline phones. The data about total population itself should be taken from other sources, such as nationwide surveys and censuses.

For the subsample of stationary phones, there are two possible ways of formation: based on telephone directories and based on a random selection of numbers (RDD). Since researchers find significant differences between the numbers included and not included in reference books, it is recommended to use the RDD method. Stationary numbers are included in the nationwide sample in proportion to their intended territorial distribution across regions of the state, and then systematic selection procedures are using (Elkasabi, 2015). As a basis for the formation of the RDD sample, a telephone directory can be used, which allows to determine the so-called active number ranges - blocks of adjoining numbers.

A subsample of mobile phone numbers is a stratified RDD sample with equal distribution among mobile operators so that all combinations of digits of numbers for all operators’ prefixes have an equal chance to get into the sample (Elkasabi, 2015). This allows representing in the starting sample different regions of a country and different social groups of subscribers, since operators most often have different areas of coverage.

However, the use of RDD-samples requires much more effort and time-consuming to get enough interviews for the analysis. Increasingly in various countries pre-dialer, which goes without a call for the subscriber, is used. In Russian practice this is often called "pinging" and is performed by special services of mobile operators.

Respondent Selection

Repeated attempts to dial all selected numbers are common practice during the survey period. Different research centers recommend a different number of attempts to dial the chosen phone numbers (from 3 to 15), but there are no methodological justifications for such recommendations. General recommendation: calls should be attempted throughout the specified period of field work.

The selection of the respondent within households in the subsample of stationary numbers is conducted in such a way as to avoid sample bias (Groves et al., 1988; Salmon & Nichols, 1983). When mobile phone surveys are conducted, as a rule respondents who answered the interviewer's call are eliminated only when they have not yet reached the age of majority or are not permanently living in the certain region (in the national-wide surveys interviews still can be conducted, and the territorial identity of the phone number can be adjusted).

Recent studies (Brick, Brick, Dipko, Pressen, Tucker & Yuan, 2007; Matthews et al., 2016) show that mobile phones, which were originally considered to be devices of individual use can also be used by several people at once. For this reason the selection of the respondent when calling to mobile numbers is also relevant as in the case of landline numbers, but is rarely used. This is a methodological problem that needs to be solved. In addition, the a priori assumption that there is only one person behind one mobile phone number is reflected in those weighing formulas that are often used in studies with DFD design (Kalsbeek & Agans, 2007), which can also distort the final results.

Studies show (Busse & Fuchs, 2013) that mobile phones are rarely used in households in the same way as stationary ones, that is, when the right to use the telephone equally belongs to all members of the household. However, researchers must consider the possibility of such cases, since the practice of using mobile communication as stationary may be characteristic of certain socio-demographic groups (for example, young and married respondents), and the characteristics of these groups can vary between countries. Similar practices of joint / equal use of a mobile phone require the same procedures for selecting respondents as in the case of surveys on landlines, and give rise to similar problems, such as the of “gatekeepers”-problem in households (Gaziano, 2005).

Interviewing

The use of mobile phones leads to measurement errors that are not typical for landline-phone surveys. In studies conducted in the 2000s, such aspects were pointed out, such as the frequent presence of respondents who were interviewed by mobile phones outside the home; finding such respondents in the presence of other people; performing other tasks during an interview (multitasking); lower communication quality and noises; opportunity for the respondent to be distracted during the interview; questions of material costs and benefits for the respondent; the difference of ideas about who and for what purpose can call the respondent on a mobile phone as a personal device, as opposed to a landline phone that is common to all members of the household; no fear of interception of conversation by members of the household on the same phone number (Lynn & Kaminska, 2013).

Along with the spread of mobile communication and the improvement of its quality, wireless landline phones are increasingly being used. And they also let to move around the house and perform various tasks during a telephone conversation. This led to the fact that now a significant difference between the interviews on stationary and mobile phones is not detected (Lynn & Kaminska, 2013). Except that mobile interview time is 10% more than stationary. This is attributed to the fact that the respondent may take time to move to a place more convenient for conversation, as well as the need to ask him additional questions (concerning, for example, his place of residence and stay) that are not required for stationary number interviews. In general, due to the proliferation of telephone surveys, interviews tend to increase up to 30–40 minutes at the stage of compiling the survey (Analytical statement, 2010).

Response, Refusal and Cooperation Rates

In the same study are different for fixed and mobile phones, since the number of required call attempts and the number of interviews that can be obtained from the first dialer are differ. But these indicators vary considerably between countries, which is associated with different practices of using telephone communications. Therefore in studies with a dual frame sample, one of the parts of the DFD design requires more time to reach the target sample size. This is an extremely important aspect of conducting field work and achieving the planned sample, since if the survey on mobile numbers goes faster than on stationary ones, then the share of the latter in the final sample will be lower than in the starting sample. The most effective way to deal with this bias is stratification of starting samples of fixed and mobile phones.

Over the past two decades, there has been a steady decline in the level of answers on both fixed and mobile phones. It is difficult to define the real reasons for the refusals, since the answers to the questions about the reason for the refusal do not reveal true motives of behavior and should be considered only as “excuses” (Brick & Williams, 2013). Often, a decrease in the response rate is explained by decrease in the level of social capital in a society that is primarily characterized by the presence of broad social connections / networks, a strong civic identity and a high level of trust in the surrounding community. Therefore, the refusal of the interview is also connected with the idea of the importance for people to take part in something that has social significance (Abraham, Maitland, & Bianchi, 2006; Brick & Williams, 2013). They are opposed by those who argue: the theory of social capital captures too wide a range of issues and it is quite difficult to verify empirically. These researchers offer a narrower and more specific theory related directly to the state of the mass survey industry (Singer & Presser, 2008; Tourangeau, 2004). Scientists point out that non-response level increases due to the widespread use of telephones for conducting mass surveys, and in particular for telemarketing. This gave rise to fatigue and irritation of people against any, including socially significant, surveys.

A more significant is the question of whether non-response are more characteristic for certain categories of the population, and, thus, whether non-responses lead to systematic errors. Are there any significant differences between those who answered interview questions and those who refused it? Some studies of the 2000s questioned the well-established view that a high response rate is a key indicator of the quality of the study (Keeter, Miller, Kohut, Groves, & Presses, 2000; Keeter, Kennedy, Dimock, Best, & Craighill, 2006; Holbrook, Krosnick, & Pfent, 2008). Comparing the results of studies where response rates differed by a factor of two, scientists did not find a statistically significant difference in the percentages of responses received.

Weighting

In the dual frame samples with overlapping design, the high probability of getting into the sample of respondents with both cellular and landline phones is corrected first. Data is weighted according to demographic indicators obtained from other sources, such as a national census or survey. Information obtained from the respondents during the interview is also used. In addition, for each part of the DFD sampling, its own weighing procedures may be required (Lohr, 2011).

According to AAPOR (Analytical statement, 2010), there is still no consensus on optimal weighing practices in dual frame RDD samples. The most common for overlapping DFD-design is algorithm, when the obtained data are subjected to stratification by the type of sampling frame (landline, mobile, dual users). Each stratum is then subjected to repeated or sequential weighing by the main variables one by one until the optimal distribution is achieved, corresponding to census data or national surveys that the researchers rely on. The main variables are gender, age, education and ethnicity / race of the respondent. When resorting to multiplicity adjustments by weighing, attention should be paid to researchers' warnings about possible distortions that may occur in cases where the standard adjustments to the “dual user” weight are added to the weight adjustment of households that have more than one fixed telephone number. Studies have shown that such a weighting done correctly does not have a significant effect on the final results of the study, therefore, such weighing, as an additional source of possible distortions, is recommended to be neglected (Merkle & Langer, 2008).

The case of Russia (“conservatives” vs. “globalists”)

Despite a significant proportion of telephone surveys (21%) among other data collection methods in Russia, academic specialists broadcast an extremely conservative view of the applicability of this method in the Russian context in the English-speaking professional environment (Header, Lehnhoff, & Mardian, 2010). The main objection of the “conservatives” is connected with the impossibility of covering the general population. According to them an obstacle to the use of telephones for nationwide polls is the low level of distribution of landline and the extreme unevenness of this distribution — a high level of telephone installation is observed only in big cities. This is not true in fact. According to statistics and polling data for 2012, the share of landline-only owners only among those surveyed using the F2F method in Russia was 6.5%, while the share of owners of cellular and landline phones was 47.5%. The proportion of owners of only a mobile number (at least one) was then more than 40% (Saponov, 2015).

Another objection concerns geographic coverage by telephone surveys. Here “methodological conservatism” leaves them only the field for regional and local research, mainly in large cities. As shown above, just the territorial localization of terminal telephone devices, especially for mobile numbers, is problematic for telephone surveys, while in implementing a nationwide sample they give significant odds to surveys at the place of residence on the diversity and dispersion of settlements across the country (Zvonovsky, 2012). Andreenkova also claims that in Russia there has never been a single directory of private fixed numbers, as in many other countries in which such a directory is a convenient sampling frame. However, the lack of a single directory does not hinder the development of telephone surveys in other countries. In addition, both from theoretical and practical points of view (see above), the use of the RDD method for sampling has significant advantages over the use of ready-made databases and was indicated as the preferred method to using directories even before mobile phones distribution (Groves et al., 1988).

Andreenkova is skeptical about the random sample of the respondent within a household based on the composition of the household (such as the Kish procedure), since questions about the composition of the household are perceived as posing a threat. Attempts to use such procedures significantly increase the failure rate. This remark is absolutely true, but equally with respect to surveys at the place of residence. There is also nothing specific in other problems named by Andreenkova: underrepresentation of men, people of 35-45 years old, youth and people with a low level of education. The same difficulties are characteristic of the F2F methods.

As if discussing with her, Saponov gives similar data on the coverage of various social groups by telephone surveys in the USA. “The share of the US non-telephoned population in 2004 was 4.9%, slightly less than the share of the non-telephone population of Russia 8 years later, in 2012, it was 5.7%. The non-telephone population in the USA is highest in the group of young people - 12.6%, in the group of Latin Americans - 10.2%, and, like in Russia, in the group of people with incomplete secondary education - 11.9%” (Saponov, 2015). The underrepresentation of certain population groups in telephone surveys is a problem faced by researchers in all countries, not only Russia. But this does not restrain the use of telephone surveys in Russian research practice, but only allows using approved solutions abroad and obtaining comparable results.