Non-Observation Bias in an Address-Register-Based CATI/CAPI Mixed Mode Survey

Abstract

Landline surveys suffer from an increasing risk of excluding a relevant share of the population. To analyze and correct telephone coverage issues, face-to-face surveys are often used, which contain questions about landline ownership and registration. Others use dual frame approaches and compare results from the landline with another mode. However, such surveys lack information about unobserved sample members.

In this article we analyze representation bias using a household survey with a sample drawn from a population register, where landline is used for households with a matched landline, and face-to-face for those without. We distinguish between the different components of nonobservation, including landline undercoverage, non-contact, and non-cooperation, by either incorporating face-to-face sample members or not, and by the fieldwork phases to recruit households and individuals. Our main interest is how biases from each of these components add up to a final representation bias in the responding sample. In addition, we analyze income and deprivation differences by either including face-to-face sample members or not.

The strongest representation bias in the telephone sample on the household level is caused by telephone undercoverage. The combined sample suffers much less from representation bias, which mostly stems from noncooperation. In terms of income and deprivation differences, our results show that the face-to-face sample is poorer than the telephone sample and needs to be considered for unbiased estimates. Based on these findings we offer some fieldwork recommendations to help reduce selection bias based on the different reasons for nonobservation.