We describe a method assessing inattentive responding in internet-based studies and explore factors associated with this problem. We inserted ‘bogus’ items into an online questionnaire completed by 210 participants and looked at associations between items and possible predictors. We found evidence of inattentive responding despite all participants being willing volunteers. Inattentive responding was related to being male and lower scores on conscientiousness, openness to experience and agreeableness. Researchers carrying out internet-based research should use methods, such as inserting ‘bogus’ items, to detect inattentive responders with a view to excluding their data.

Introduction

Although using the internet to overcome recruitment barriers is common in mental health research, the practice is criticised due to the lack of environmental control compared to face-to-face methods (Roivainen, Veijola, & Miettunen, 2016). Inattentive responding has been identified as a particular problem and is defined as responding without regard to item content (Huang, Bowling, Liu, & Li, 2015). People may randomly choose from all response options on a scale, or employ a non-random pattern (e.g. scoring all items the same). Estimated rates of inattention have varied widely, from 3% to 46% of respondents due to lack of clarity on how to define and measure it (Meade & Craig, 2012). Inattentive responding constitutes error variance, which attenuates correlations, reduces internal consistency reliability estimates and leads to erroneous factor analytic results. Common recommendations for cleaning problematic responses, such as univariate outlier analysis rest on the assumption that careless or inattentive responses are rare or extreme in magnitude (Meade & Craig, 2012). Meade and Craig (2012) assessed the utility of different measures and recommended the use of ‘bogus items’, which are improbable statements that only have one correct response. Incorrect answers therefore indicate inattentive responding. Such statements might include, “I drive in reverse on the motorway”. Such items are easy to create and provide an obvious metric for scoring as correct or incorrect. Unlike other approaches that can be cumbersome to implement (e.g. psychometric antonym) or are format-dependent (e.g. long strings), the bogus item approach is a simple method that can be used across different surveys (Huang, Bowling, Liu, & Li, 2015).