Measuring Church Attendance: Phone Calls vs. Online Surveys

Many national surveys are done via telephone, with the responded answering questions verbally through the phone when asked a specific question.  However, over time people have become less likely to answer requests to answer phone-administered surveys.  An alternative is to try online surveys in which the respondent does not interact with another person live but rather just clicks button using an online form.  However, while some people prefer the phone surveys, others prefer the online survey format, and when these people differ in characteristics that also are correlated with religious behavior, then the choice of survey format can lead you to mis-estimate the religiosity of the population.

A report just published by the Pew Research Center discusses some work they have done comparing the responses of different survey formats and then reweighting responses to obtain what should be more accurate overall measures of American religiosity.  See the front page here and the full report here.

They report some key findings:

• Phone surveys accurately measure religious affiliation, while online surveys underreport affiliation.
• Phone surveys overstate church attendance rates to a large degree, while online surveys are more accurate, maybe with just a slight understanding of attendance..
• Phone surveys overstate and online surveys slightly understate the share of the population who think religion is important in their lives.

These findings are not too surprising given that older respondents are generally more religious than younger ones and that the older respondents prefer phone surveys more than the younger ones and the younger ones prefer online surveys more than the older ones.  Reweighting responses is an important aspect of survey methodology that enables researchers to improve the accuracy of their findings.