How to increase the acceptance of mobile app and sensor data collection?
Download data and study materials from OSF
Principal investigator:
Alexander Wenz
University of Mannheim
Email: a.wenz@uni-mannheim.de
Homepage:
https://www.wenzalexander.com/
Sample size: 1876
Field period: 04/27/2021-08/24/2021
Abstract
To study human behavior, social scientists are increasingly collecting data from mobile apps and sensors embedded in smartphones. A major challenge of studies implemented on general population samples, however, is that participation rates are rather low. While previous research has started to investigate the factors affecting individuals’ decision to participate in such studies, less is known about features of the study design which are under the researcher’s control and can increase the acceptance of smartphone-based data collection methods. Guided by the Technology Acceptance Model, we varied study characteristics in a vignette experiment to examine their effect on individuals’ willingness to download a research app on their smartphone. Data were collected from 1,876 members of the NORC AmeriSpeak Panel, a probability-based panel of the general population aged 18+ in the United States. Respondents were randomly assigned to eight vignettes and, after each vignette, were asked to rate their willingness to participate in the described hypothetical study. The results show that individuals are more willing to participate in smartphone-based studies where they have some control over the data collection process, either by having the option to temporarily switch off the data collection or to review the data before submission. Furthermore, they are more willing to participate in research to which they are invited via postal letter rather than receiving a postal letter plus a phone call from an interviewer who walks them through the app installation. Finally, unconditional incentives increase their willingness to engage with smartphone-based data collection over conditional incentives.
Hypotheses
- H1.1. Providing simpler data protection and privacy statements in the study invitation increases individuals’ willingness to participate in mobile app and sensor data collection (WTP).
- H1.2. Providing simpler data protection and privacy statements in the study invitation has a larger positive effect on WTP for individuals with higher levels of privacy concerns compared to those with lower levels of concerns.
- H1.3. Providing simpler data protection and privacy statements in the study invitation has a larger positive effect on WTP for individuals with lower educational attainment compared to those with higher educational attainment.
- H2.1. Providing technical support during app installation increases individuals’ WTP.
- H2.2. Providing technical support during app installation has a larger positive effect on WTP for individuals with lower levels of smartphone proficiency compared to those with higher levels of proficiency.
- H3.1. Giving study participants more control over their data during data collection increases individuals’ WTP.
- H3.2. Giving study participants more control over their data during data collection has a larger positive effect on WTP for individuals with higher levels of privacy concerns compared to those with lower levels of concerns.
- H4.1. Providing unconditional monetary incentives increases individuals’ WTP over conditional monetary incentives.
- H4.2. Providing conditional monetary incentives paid as incremental amounts increases individuals’ WTP over those paid as fixed amounts.
- H4.3. Providing personalized feedback as a non-monetary incentive increases individuals’ WTP over no feedback.
Experimental Manipulations
The vignettes contained descriptions of hypothetical studies that would invite respondents to download a research app on their smartphone. Respondents who reported not using a smartphone were informed that they would be provided with a device for the purpose of the study. The research app would administer survey questions about individuals’ time use and collect data about the technical characteristics of their phone, whether their phone is currently in motion, their current location, what apps are used and what websites are visited, as well as the number of incoming and outgoing phone calls and text messages on their phone.
Five study characteristics were randomly varied in the vignettes:
- Data protection and privacy statement
Easy
The study would be in line with U.S. regulations about data protection and privacy. All information collected by the research app would be confidential. It would only be used by the researchers and they would not share your individual data with anyone else. The researchers would write publications about the overall results of the study. They would also present the findings at international conferences.
Difficult
The study would be in compliance with prevailing U.S. data protection and privacy regulations and several procedures would be in place to preserve the confidentiality of all information collected by the research app. Any data collected would only be used by the principal investigators and would not be shared with any other parties. Aggregated study findings would be disseminated through publications and international conferences.
- Technical support during app installation
Yes
You would receive a letter and a phone call from an interviewer who would walk you through all steps to install the app. They would also show you how to use the app and answer any questions you may have.
No
You would receive a letter with login details and a link to the app store where you could install the app. The letter would also contain instructions on how to use the app.
- Control over data collection
Turn off data collection
The research app would allow you to switch it off at times when you do not want it to collect any data from your smartphone. However, it would be important for this study that you use this option only in rare cases and for short periods.
Review data before submission
The research app would allow you to review the data that have been collected and you would have the option to delete specific data points when you do not want them to be transmitted to the researcher. However, it would be important for this study that you use this option only in rare cases.
No
There would be no option to switch the research app on and off during the course of the study.
- Monetary incentive
Unconditional
You would receive $20 enclosed in the invitation letter as a token of appreciation for your time.
Conditional with incremental amount
You would receive $6 for downloading the app and $1 for every day that you leave the app installed on your smartphone, resulting in a maximum amount of $20.
Conditional with fixed amount
You would receive $20 for downloading the app and leaving it installed on your smartphone until the end of the study.
- Feedback
Yes
You would also receive a personal summary of your results at the end of the study.
No
–
Outcomes
Willingness to download research app (“How likely is it that you would download the app to participate in this research study?”) on an 11-point scale (“0 Definitely would not participate” to “10 Definitely would participate”)
Summary of Results
- H1.1. Providing simpler data protection and privacy statements in the study invitation has no significant effect on individuals’ WTP (p=0.267), not supporting H1.1.
- H1.2. The interaction effects of the simplicity of the data protection and privacy statement and the level of privacy concerns on individuals’ WTP are not significant (p=0.124 for security concerns towards research apps; p=0.471 for number of perceived privacy violations offline; p=0.468 for number of perceived privacy violations online; p=0.082 for trust in organizations involved with mobile data collection), not supporting H1.2.
- H1.3. The interaction effects of the simplicity of the data protection and privacy statement and the level of educational attainment on individuals’ WTP are not significant (p=0.655 for high school degree vs. no high school degree; p=0.651 for college degree vs. no high school degree), not supporting H1.3
- H2.1. Providing technical support during app installation significantly decreases individuals’ WTP (p<0.001), not supporting H2.1.
- H2.2. The interaction effects of technical support during app installation and the level of smartphone proficiency on individuals’ WTP are not significant (p=0.218 for smartphone use vs. non-use; p=0.983 for frequency of smartphone use; p=0.163 for number of smartphone activities), not supporting H2.2.
- H3.1. Giving study participants more control over their data during data collection significantly increases individuals’ WTP (p<0.001 for turn off data collection; p<0.001 for review data before submission), supporting H3.1.
- H3.2. The interaction effects of control over data collection and the level of privacy concerns on individuals’ WTP are significant for one of the indicators of privacy concerns (p<0.001 review data before submission X number of perceived privacy violations offline) but in a different direction than expected; the interaction effects are not significant for the other indicators of privacy concerns (p=0.949 for turn off data collection X security concerns towards research apps; p=0.892 for review data before submission X security concerns towards research apps; p=0.220 turn off data collection X number of perceived privacy violations offline; p=0.450 turn off data collection X number of perceived privacy violations online; p=0.925 review data before submission X number of perceived privacy violations online; p=0.198 turn off data collection X trust in organizations involved with mobile data collection; p=0.485 review data before submission X trust in organizations involved with mobile data collection), not supporting H3.2.
- H4.1. Providing unconditional monetary incentives significantly increases individuals’ WTP over conditional monetary incentives (p=0.010 for conditional with fixed amount vs. unconditional; p<0.001 for conditional with incremental amount vs. unconditional), supporting H4.1.
- H4.2. Providing conditional monetary incentives paid as incremental amounts significantly decreases individuals’ WTP over those paid as fixed amounts (p=0.008), not supporting H4.2.
- H4.3. Providing personalized feedback as a non-monetary incentive has no significant effect on individuals’ WTP over no feedback (p=0.531), not supporting H4.3.
References
Wenz, A. & Keusch, F. (in press). Increasing the acceptance of smartphone-based data collection. Public Opinion Quarterly.
Wenz, A. & Keusch, F. (2022). “How to increase the acceptance of mobile app and sensor data collection?” Paper presented at the General Online Research Conference, Berlin, Germany, September 7-9, 2022.
Wenz, A. & Keusch, F. (2022). Wie lässt sich die Akzeptanz von Smartphone-Datenerhebungen erhöhen? [How to increase the acceptance of smartphone-based data collection?] Paper presented at the Spring Conference of the Section “Methods of Empirical Social Research” of the German Sociological Association, Virtual, March 25-26, 2022.