Evaluation of an Adapted Design in a Multi-device Online Panel: A DemoSCOPE Case Study

Abstract

In this paper, we look at the challenge of optimizing survey layout in online research to enable multi-device use. Several studies provide useful advice on target-oriented implementation of web design for CAWI surveys. This paper presents results of the implementation of a new adapted design at the panel of DemoSCOPE that allows the participants to take part in a survey on multiple (especially mobile) devices. To evaluate this adapted design, we compare interview data and question timing of panellists who participated in an insurance study before and after the design transition. Central key figures concerning the completion rate, item non-response, open questions, straightlining, timing of single questions and the length of the total interview are presented. In addition, we have presented examples of both old and new design to the community and invited them to assess these examples concerning orientation, color, design and usability. We evaluate the differences in these assessments before and after the design transition for smartphone and desktop users. We end with suggestions for best practice for online studies on different devices.