Pew Religious Knowledge Survey

The Pew Forum just released results from its Religious Knowledge Survey. See here for details. It's getting a lot of buzz in newspapers and online media sources because atheists and agnostics scored highest on average overall, thus suggesting that atheists and agnostics know more about religion than religious people.

But this way of summarizing the data is very misleading because it implicitly places specific weights on different types of religious knowledge, and people may disagree on how these weights are placed. The simple average gives each question equal weight, but the distribution of questions is certainly arbitrary. Thus, an overall score is an arbitrary measure of religious knowledge and not very insightful. Think about it. What if you removed a couple questions about the Bible? How would that change the overall scores? What if you added a couple questions about the Bible?

A better way to look at the data is to look at topic specific questions. Look at Bible questions, then world religion questions, then public policy and religion questions. Check the scores for the people in different religious groups by type of question. This is more informative. Alas, when you do this you find that that the results are not that surprising if you already know something about the people in these religious groups.

P.S. You can test your own knowledge on this reduced version of the survey. I got 15 out of 15 correct. How many did you get correct?