Many national surveys are done via telephone, with the responded answering questions verbally through the phone when asked a specific question. However, over time people have become less likely to answer requests to answer phone-administered surveys. An alternative is to try online surveys in which the respondent does not interact with another person live but rather just clicks button using an online form. However, while some people prefer the phone surveys, others prefer the online survey format, and when these people differ in characteristics that also are correlated with religious behavior, then the choice of survey format can lead you to mis-estimate the religiosity of the population.
A report just published by the Pew Research Center discusses some work they have done comparing the responses of different survey formats and then reweighting responses to obtain what should be more accurate overall measures of American religiosity. See the front page here and the full report here.
They report some key findings:
- Phone surveys accurately measure religious affiliation, while online surveys underreport affiliation.
- Phone surveys overstate church attendance rates to a large degree, while online surveys are more accurate, maybe with just a slight understanding of attendance..
- Phone surveys overstate and online surveys slightly understate the share of the population who think religion is important in their lives.