"I actually agree that it's meaningless, and I'll explain my position. Simply saying "scientists" is simply too broad of a term. Most of the members of M.U.F.O.N. are considered "scientists." However, they're also nut jobs."
---Here's a quote from the Pew website as to how they conducted their poll:
About the Scientist Survey
Results for the scientist survey are based on 2,533 online interviews conducted from May 1 to June 14, 2009 with members of the American Association for the Advancement of Science (AAAS), under the direction of Princeton Survey Research Associates International. A sample of 9,998 members was drawn from the AAAS membership list excluding those who were not based in the United States or whose membership type identified them as primary or secondary-level educators.
Founded in 1848, AAAS is the world’s largest general scientific society, and includes members representing all scientific fields. AAAS publishes Science, one of the most widely circulated peer-reviewed scientific journals in the world. Membership in AAAS is open to all.
Each person sampled was mailed a letter on stationery with logos of both the Pew Research Center for the People & the Press and AAAS. The letter was signed by Andrew Kohut, President of the Pew Research Center and Alan I. Leshner, Chief Executive Officer of AAAS. These letters were intended to introduce the survey to prospective respondents, describe the nature and purpose of the survey and encourage participation in the survey. The advance letter contained a URL and a password for a secure website where the survey could be completed. The letter also included a toll-free number for respondents to call if they had questions.
http://people-press.org/reports/images/528-101.gifSubsequent requests to complete the survey were sent to those who had not yet responded. These requests were sent by e-mail for those who could be contacted this way (three e-mail reminders were sent) and by postal mail for members who had told AAAS they preferred not be contacted by e-mail (a postcard and letter reminder were sent).
A total of 1,411 of the 5,816 sampled members in the e-mail group completed the interview for a response rate of 24%. In the mail group, 1,122 members of the 4,182 sampled completed the survey for a response rate of 27%. The overall response rate for the study was 25% (2,533 completes/9,998 sampled members). Nearly all respondents completed the survey online; however, a very small number requested to complete the survey in another mode; twenty interviews were completed by telephone.
Nonresponse in surveys can produce biases in survey-derived estimates because participation may vary for subgroups of a population, who may differ on questions of substantive interest. In order to correct for these biases, weighting is often employed.
To evaluate the possibility of nonresponse bias in the scientist survey, respondent characteristics from the obtained sample were compared with known characteristics of the population, based on membership and demographic information in the AAAS membership database. For most characteristics the sample was very representative of the population of all members. The most notable differences were that the sample underrepresented student members and overrepresented those with emeritus status. There also were differences in response rates between those who could be contacted by e-mail and those for whom no e-mail address existed or e-mail contact was not permitted. To correct these potential biases, the data were weighted so that the sample matched the two parameters of contact mode and member category from the AAAS membership database.The following table shows the error attributable to sampling that would be expected at the 95% level of confidence for different groups in the scientist survey:
