An Empirical Study of Evolution, Creationism, and Intelligent Design Instruction in Public Schools

By Bowman, Kristi L

ABSTRACT Recent reports from students and teachers alike suggest that even though teaching creationism as a credible scientific theory in public schools is clearly unconstitutional, some teachers still do just that. Conversely, despite the fact that many states’ educational standards explicitly require that evolution be taught, anecdotal evidence also suggests that evolution sometimes is not discussed at all or, more frequently, may be the subject of mere cursory instruction. But, how widespread are these practices? This Article reports and analyzes the results of a 2006 survey in which nearly 1,000 college students from across the country provided their recollections about the frequency and manner of evolution, creationism, and intelligent design instruction in their high school science classes. In sum, about three out of ten recent public high school graduate respondents recalled that they were taught about creationism in science class and two out often reported receiving intelligent design instruction, although not all instruction communicated that a given concept was scientifically credible. Nearly all recent public high school graduate respondents reported receiving some evolution instruction (92%), but surprisingly few (73%) received much of it. Additionally, the results of this survey show that cultural factors such as politics and geographic region are somewhat more strongly correlated with disparities in perceived classroom instruction than are states’ educational standards; the rate of disregard for constitutional principles in this context is low but definitely not nonexistent; and legal literature does not sufficiently address constitutionally problematic practices occurring in classrooms.

I. INTRODUCTION

The scene is not difficult to picture: it is May-this year, ten years ago, five years hence-and as the school year draws to a close, a public high school biology teacher skims over the theory1 of evolution in class, perhaps following a suggestion from the principal or a more formal direction from the local school board or the state board of education, maybe afraid to invoke the ire of some students or parents (or just too worn down to deal with the potential fallout), or possibly playing out his or her own skepticism of Charles Darwin’s big idea. For these reasons or others, the teacher may go further still and present creationism or intelligent design as a scientifically valid alternative to evolution. Variations of this story, by now, are the subject of frequent anecdotes.2 But, a dearth of research exists in this area, particularly regarding the perspectives of the students themselves who currently attend or recently graduated from public high schools across the country, and so we have little idea whether those anecdotes are merely isolated, decreasingly frequent instances, or whether they are reflective of widespread contemporary practices.

This lack of information has impacted legal scholarship. Admirable works in this area have engaged complex questions of the constitutionality of various practices, questioned the propriety of different Establishment Clause tests or factors, and considered ways in which science and religion can be compatible.3 However, because our sense of what happens in public high school science classrooms often is based more on conventional wisdom than anything else, this scholarship largely has been unable to situate the issues it discusses in the relevant social context by making reference to the magnitude of any particular classroom practice identified as a “problem.” Thus we don’t know to what extent the existing scholarship addresses pervasive practices in our schools or relatively infrequent ones, and to what degree Establishment Clause doctrine has consistently filtered down into classrooms. Stated more bluntly, we have not had the data that will allow us to discuss in a comprehensive way how frequently public high school science teachers broach the topics of evolution, creationism, or intelligent design in a manner that violates well-settled, or emerging, dictates of constitutional law or state regulation.

In a practical sense, whether any illegal behavior individual students observe is the only such instance or one of many is irrelevant: according to the Supreme Court, it is in public schools that students are supposed to learn to become citizens, and observing a government actor’s disregard for law may well be memorable, though perhaps not the sort of civics lesson we want to impart.4 Therefore, sporadic law-breaking behavior by government actors certainly is not inconsequential. However, such episodic behavior differs materially from a persistent pattern of illegal practices across school districts or even states because a pattern is more likely to suggest either that some (perhaps many) government actors do not understand what the law requires, that communities tolerate government actors’ flouting of this area of the law, or both. And, with the type and degree of purported political and regional divisions within the United States,5 it seems plausible that the level of tolerance for disregarding constitutional dictates or state educational standards could vary depending on a state’s partisan climate and on the area of the country in which a given state is located.* Such variance would seem troubling, to say the least-and a fitting subject of academic inquiry.

Additionally, legal scholarship in this area has only begun the important task of engaging the question of how the Establishment Clause’s general prohibition of government’s “hostility towards religion” could or should apply in the elementary and secondary public school context, and specifically when instruction critical of creationism or intelligent design occurs.7 This situation in a science class is more difficult than it may first appear because a science teacher’s overt criticism of “creation science,” a clearly religious concept, could well be grounded in his or her scientific training and earnest desire for students to learn traditional methods of scientific inquiry and analysis, and also be completely unrelated to any desire to criticize religious beliefs. Yet, the purpose apparent to students or the effect experienced by them could be quite different. Perhaps academic discussion of this topic has not been more robust because there are few, if any, cases presenting such a factual scenario; the absence of such a situation from the pages of the Federal Reporter to date, however, certainly does not equate to its absence from classrooms. Thus, having a better understanding of just what is happening in public high school science classes across the country, and how students perceive that government action, also allows legal scholarship to better engage the unresolved legal issues waiting just offstage.

For these reasons and others, we should have a much better idea than we do about how frequently, and in what manner, evolution, creationism, and intelligent design are taught in public high school science classrooms across the country. Yet, what is the best way to ascertain this information? Measuring the evolution-related instruction occurring in public school science classes directly or indirectly is incredibly difficult, which seems likely to be a major reason why quantitative research on this subject is scarce.8 Setting aside opinion polls about what should be taught in public high school science class,9 the science and social science pieces that focus on what actually is taught are of limited utility: the studies concentrate on individual states and, because different survey instruments are used, drawing cross-state comparisons is quite difficult; not all of the research is current-many of the studies occurred over twenty years ago and only a small minority occurred within the past five years; and nearly all studies obtained their data by surveying science teachers, often at state science teacher meetings, whose occupation alone introduces particular types of bias to those studies.10 Almost no research has been conducted regarding students’ impressions of these same issues-and none yet has focused on comparing the experiences of students across the country.11

Student opinion is of increasing import for various reasons. Not only do students provide valuable perspectives on events that are not feasible for a researcher (or even a team of researchers) to measure directly,12 but students also are the subjects of education. As a result, to the extent that perceived purpose, perceived or actual effect, apparent endorsement, and/or coercion are portions of an Establishment Clause analysis,13 students’ perspectives about the instruction they received matter greatly when evaluating whether the requisite level of constitutional harm has been met.

Therefore, this Article presents and analyzes the descriptive statistical14 results of a 2006 study in which almost 1,000 college students from across the country completed a survey asking about the frequency and manner of evolution, creationism, and intelligent design instruction in their high school science classes.15 (Because of the limitations of the data,16 readers are cautioned against extrapolating the results nationally.) According to the nearly 600 respondents who were recent public high school graduates in one of the eight states selected as the focus of the study, nearly all of them received some evolution instruction (92%), but less than three- quarters of them (73%) were taught evolution in any great detail, and only 69% of them perceived that evolution was presented as a credible scientific theory.17 (Evolution was defined as “the theory that all species evolved from less complicated organisms, and that individual species change over time[,] most closely associated with Charles Darwin.”18) By comparison, 30% of the recent public high school graduates reported receiving some type of creationism instruction, although the vast majority of that instruction reportedly consisted of a brief mention of the concept. Furthermore, only 6% of them reported that creationism was taught as a credible scientific theory, and 13% reported that creationism was taught as lacking scientific credibility-hence, understanding the manner of instruction (the degree of perceived scientific credibility) gives much greater meaning to the “frequency of instruction” data. (Creationism was defined as “the idea that God created all living and non-living things in more or less present form, and that humans and apes do not share a common ancestor.”19)

Intelligent design-a concept whose recent popularity dates to the late 1980s and early 1990s20-was defined in the survey as “the idea that the complexity of living things, and the low probability of evolution producing such complexity, can be explained only by the existence and involvement of an intelligent designer.”21 Reportedly, this concept was taught in the science classes of 19% of recent public high school graduate respondents; like creationism, most of this instruction consisted of a brief mention.22 If the legal reasoning in the nation’s first intelligent design case is determinative and the answers provided by survey respondents are accurate, then unconstitutional intelligent design instruction occurred in 7% of recent public high school respondents’ science classes.23

Although the legal status of intelligent design is unclear, the law regarding teaching creationism and evolution was clear when respondents were enrolled in high school biology class, as it is now. Twenty years ago, in 1987, the Supreme Court held in Edwards v. Aguillard that teaching “creation science” violates the Establishment Clause; this case remains good law.24 Correspondingly, since 2000, the eight states that are the subject of this study have maintained consistent evolution-related educational standards. Some of these eight states and others require that evolution be taught in depth, while others do not even use the “e” word in their educational standards, and many more fall somewhere in the middle.25

Readers interested primarily in the presentation of data are advised that the discussion of general results begins at the end of Part III. Much must occur before then, however. The next section of this Article, Part II, discusses legal rules governing evolution, creationism, and intelligent design instruction (state standards and the Establishment Clause), and addresses the relevance of two additional social factors by which the data are disaggregated in a later portion of the Article (states’ Red-Blue dominant political climates and states’ regional locations). Part III then turns to the survey’s methodology, also discussing potential methodological challenges, and presents the demographic profile and composite responses of recent public high school graduates regarding the frequency and manner of evolution, creationism, and intelligent design instruction in their high school science classes.26 Part IV examines the data further by evaluating reported evolution, creationism, and intelligent design instruction in turn according to three disaggregated categories: respondents who attended public high school in (1) states with weak versus strong evolution-related educational standards, (2) Red versus Blue states, and (3) states in the Northeast, South, Midwest, and West. The corresponding results of logit regression analyses are presented in Part IV when they are statistically significant. As such, Part IV evaluates the strength of any connections between various legal or social factors and reported evolution, creationism, or intelligent design instruction.

II. THE LEGALAND SOCIAL CONTEXT OF EVOLUTION, CREATIONISM, AND INTELLIGENT DESIGN INSTRUCTION

This section addresses the various legal and social factors that may be connected with evolution, creationism, and intelligent design instruction in public high school classrooms, and which constitute a central focus of this study.

A. Legal Context: State Standards and the Establishment Clause

Evolution instruction is governed most formally by states’ educational standards. Accordingly, the survey was designed so that the responses of students who attended four years of public high school in states with strong evolution-related science standards could be compared with those who attended four years of public high school in states with weak evolution-related standards. As a necessary prerequisite to understanding those data, this section first addresses what educational standards are, and also how the No Child Left Behind Act of 2001 (“NCLB”)27 impacts the development of science standards as well as the content of in-class instruction. Additionally, because the survey evaluates the frequency and manner of creationism and intelligent design instruction, matters regulated by the Establishment Clause, this section sets forth an overview of the ways in which Establishment Clause doctrine can be relevant to instruction regarding those topics.

1. State Standards: Administrative Regulation of Educational Content28

It is no secret that American high school students lag behind their counterparts worldwide in science and math achievement.29 State and federal educational bureaucracies have been attempting to narrow this gap in part by relying on the educational reform model de jour: standards-based testing as elaborated in NCLB.30 In contrast to standardized tests which test general skills and knowledge (e.g., the SAT, ACT, and LSAT), standards-based tests are unique to each state because they assess students’ skills and knowledge as compared to particular, statedefined educational outcomes (standards) intended to define a core set of ideas and information public school students should learn in various subjects and at different grade levels.31 Standards do not establish a statewide curriculum in that they do not dictate which textbooks must be used or which assignments or methods of in-class assessment must be employed, yet they do establish a substantive instructional framework. The content of a state’s standards explicitly is intended to influence the instruction students receive, and to do so at a low cost, compared to many other methods of educational reform.32

States’ science standards range in length from barely a dozen pages to more than ten times that and contain varying levels of detail.33 For example, in Ohio, standards are lengthy, specific, and accompanied by model lesson plans;34 in other states, such as Montana, standards are incredibly general and brief.35 In 44 states, it is the state board of education that functions as an administrative agency rule-maker and establishes these educational standards.36 Although nearly all states had developed science standards before 2001,37 NCLB required that all states have science standards by the 2005-2006 school year38 and begin “assessing science learning” (in other words, begin administering standards- based science tests) by the 2007-2008 school year.39

The enforcement of state educational standards-here, the required teaching of evolution in some states-is rather different from the enforcement of individual rights or statutory prohibitions. State standards do not vest students with rights, provide for a private right of action to enforce the teaching of state standards, or even carry specific penalties. Instead, standards compliance reasoning generally goes as follows: if a teacher does not teach to the standards, then his or her students are more likely to perform poorly on the later standards-based tests; if a subgroup or composite of students does not demonstrate the requisite level of proficiency on a state’s standards-based test, then the slow guillotine of NCLB reorganization and restructuring will be set in motion.40 (Clearly, though, students’ test performance is influenced by many other factors, making this enforcement “mechanism” weak, indeed.) And, because the science standards have not consistently been the subject of standards-based tests (as mentioned previously, science tests are not required until the 2007-2008 academic year), even this tenuous incentive for complying with state standards is just now coming into play.

Although teachers who disregard state standards may face disciplinary action by a principal or district administrator, actual in-class monitoring of specific curricular instruction is limited. If a teacher disregards merely one part of the subject-matter standards (such as the requirement that evolution be taught), such action would be difficult for a school administrator to identify unless the teacher discussed his or her decision with others, or students or parents complained.41 Thus the deterrents for disregarding state evolution-related education standards are weaker, and the remedies of disciplinary action and administrative proceedings less likely to be triggered, than in situations where individual rights are abridged.

Finally, in the context most relevant to this Article, states’ science standards are limited to regulating evolution instruction- they cannot promote creationism instruction without running afoul of the Establishment Clause,42 and with the exception of recent standards battles in Kansas and Ohio, state standards have not implicitly made room for intelligent design instruction, much less promoted such instruction explicitly.43 2. The Establishment Clause As Applied to Creationism and Intelligent Design Instruction

What may be most clear about current Establishment Clause doctrine is in fact its lack of clarity.44 Because several Establishment Clause tests have emerged in the context of public secondary school instruction, courts evaluating Establishment Clause questions are wise to adopt a “belt and suspenders” approach regarding the relevant tests, as the district court did in Kitzmiller v. Dover,45 the recent intelligent design case arising out of Pennsylvania. This subsection provides a skeleton outline of the various tests before turning to a brief discussion of the specific legal issues that can arise when creationism and intelligent design are presented in a public high school science classroom.

a. The Establishment Clause Tests

First, the baseline of contemporary Establishment Clause doctrine is the “Lemon test;” the subject of many a pun, the test’s namesake is merely the 1973 originating case, Lemon v. Kurtzman. The Lemon test requires that for a government action to be permissible (in other words, to comply with the Establishment Clause), the government action “[fjirst . . . must have a secular legislative purpose; second, its principal or primary effect must be one that neither advances nor inhibits religion . . . [third, it] must not foster ‘an excessive government entanglement with religion.”46 The first two prongs of Lemon (purpose and effect) have remained part of various Establishment Clause analyses, but the third prong (entanglement) is practically forgotten, although it has not been officially, uniformly disregarded. The Lemon test lacks the full support of today’s Court, which applies the Lemon factors somewhat haphazardly,47 but Lemon has not been explicitly overruled and thus remains the starting point for an Establishment Clause analysis by most lower courts.48

Second, an outgrowth of the Lemon test, the “endorsement test” focuses on whether the reasonable or objective observer (sometimes the child in school, sometimes an adult community member, nearly always omniscient), would perceive the government action at issue to constitute an endorsement of religion. The endorsement test was introduced in a concurrence by Justice O’Connor in the Court’s 1984 decision Lynch v. Donnelly49 and gradually gained the acceptance of the Court, being adopted more or less in full in Santa Fe School Independent School District v. Doe50 in 2000 and Zelman v. Simmons- Harris51 in 2002. It remains unclear whether the endorsement test was intended to supplant Lemon, or whether it should function as a supplement to the effect prong.52

Third, and separately, the “coercion test” appeared to constitute the basis for the Court’s 1992 decision in Lee v. Weisman, which declared a nonsectarian prayer at a high school graduation unconstitutional because of students’ inability to leave or remain seated without being stigmatized by their classmates. Thus, the Court held, students were coerced to participate in an action respecting religious practice.53 The Court has not rescinded the coercion test, and also has not employed it with full force since then.54

Other tests or considerations have been introduced in concurrences and dissents,55 and in contexts different from public elementary and secondary schools;56 because these ideas have not yet gained the full support of the Court in general and/or in the public school context, they are not discussed here.

b. Teaching Creationism or Intelligent Design as Scientifically Credible Theories in a Public High School Science Classroom

Despite this doctrinal discord, it is well-settled law that even when creationism is taught as “creation science,” its presentation as a credible scientific theory in a public high school classroom runs afoul of the Establishment Clause-this is the holding of the Court’s 1987 decision in Edwards v. Aguillard. In that case, the Court invalidated the state statute at issue because of the legislature’s clearly religious purpose in requiring “creation science” instruction when evolution was taught.57 Thus, to the extent that instruction supportive of creationism is required by a state or local government, or even taught by a public school science teacher at his or her own initiative, such instruction violates the Establishment Clause. The same appears true regarding intelligent design, to the extent it is considered to be a religious concept.58

Because of intelligent design’s apparent religious roots,59 the legal issues regarding intelligent design instruction have been approached in the same way, conceptually, as the creationism instruction cases. Only one case about intelligent design instruction in a public school science class has been litigated. That case began in 2004 when the Dover, Pennsylvania school board adopted a resolution directing science teachers to provide basic information about intelligent design as an alternative to evolution when teaching the evolution unit.60 The Dover school district was the first in the nation to take this step,61 and a federal district court struck down the school district’s resolution. According to that court’s December 2005 opinion, “intelligent design is a religious view, a mere relabeling of creationism, and not a scientific theory” and thus cannot be taught as a scientifically credible theory in science class without violating the Establishment Clause.62 In February 2006, the parties agreed that the small, rural Dover school district would pay $1 million of the plaintiffs’ attorney fees, which actually ran to about twice that amount.

In the year leading up to the Dover school district’s trial, 2005, 50 local school boards and 14 state legislatures considered proposals to require or specifically permit public high school science teachers to engage in a critical teaching of evolution or an affirmative teaching of intelligent design.63 Since then, few other districts have been willing to follow Dover’s lead.64 Efforts to promote intelligent design instruction via state legislatures also have been more limited.65 Additionally, in November 2006, state board members known to be intelligent design proponents were voted out of office in Kansas and Ohio after bitter, divisive campaigns.66 But, evolution advocates perceive the intelligent design battles to be far from over.67

c. Criticizing Creationism or Intelligent Design in a Public High School Science Classroom

A previous subsection discusses the constitutional tests that are most often considered to be the focus of a legal analysis of creationism or intelligent design instruction. In different ways, those constitutional tests all ask whether the government is supporting religion; not surprisingly, legal scholarship, too, has focused on the same question. But, a teacher who presents creationism or intelligent design in a public school science class does not always present either concept in a scientifically favorable light. A teacher’s criticism of either concept also could give rise to an Establishment Clause violation arising out of an unclear area of the doctrine in which the few claims brought have not met with success-government hostility to religion.68 (Traditionally, claims in which individuals grieve government action that interferes with their own religious belief or practice are brought under the Free Exercise Clause which employs a limited and also practical remedy, usually exemption, in the case of such a violation.69)

It is easy to set forth a rule that instruction critical of religion is hos tile to religion but instruction critical of science is not. Nevertheless, in the context of evolution, creationism, and intelligent design, these issues are unusually closely connected. Their entanglement may be most acute within the concept of intelligent design. Intelligent design proponents posit scientifically-couched claims that (1) the irreducible complexity of certain organisms and/or (2) the prohibitively-low odds of a random process producing the complex, varied organisms that exist today can be explained only by the existence of an intelligent designer.70 Both of the intelligent design arguments lead up to the conclusion that an intelligent designer must exist and must have been involved in the creation and diversification of the species in a certain way. Thus, if one assumes the intelligent designer is the Judeo- Christian God and not a space alien or time-traveling cell biologist (options offered by intelligent design advocates), then the truth of certain religious beliefs depends directly on the validity of the scientifically-couched claims of intelligent design.71

The majority of scientists assail intelligent design as “bad science,”72 and the federal district court to hear the one and only intelligent design case so far concluded that intelligent design was a religious concept.73 But, if intelligent design is inherently religious, does it somehow gain additional constitutional protection in science class-protection it would not have as science alone- because any criticism of it constitutes hostility towards religion? Particularly because intelligent design advocates repeatedly insist that their approach is not religion-based,74 this would seem a bizarre result and one that would win them the proverbial battle but lose them the war. Returning to the original question, though, how should a concept like intelligent design be criticized by science teachers who wish to do so, without its religious aspects also becoming a target?75 For a teacher to expressly state that intelligent design advocates might have the right result though reach it through scientifically-flawed reasoning would avoid the religious hostility problem, but of course could run into the more common endorsement problem. To focus on intelligent design’s specific arguments and to say nothing about its ultimate claims may be perceived either as favoritism or hostility towards any religious ideas, depending on many factors such as the rest of the discussion, other class sessions, and even the teacher’s tone-but, saying nothing about the ultimate claims may well be the safest option from the perspective of minimizing a school district’s legal risk. The factual issues in such a case would be thorny, and the doctrinal issues deserve much more extensive consideration.76 The likelihood of such a situation giving rise to a cognizable hostility-based Establishment Clause claim is unclear, although this may be more likely given the tenuous nature of the doctrine and recent changes to the composition of the Court.77

B. Social Context: States’ Dominant Political Climates and States’ Regional Locations

In the context of evolution, creationism, and intelligent design instruction, anecdotal evidence and some empirical evidence (particularly surveys of science teachers) suggest that a gap exists between clearly established legal requirements or prohibitions, and students’ reported experiences. Such a gap could be created in part because of an uncertainty as to what the law requires, although in the case of evolution and creationism instruction, the law is clear. It also could be created because enforcement mechanisms are not employed consistently and effectively; when the burden of enforcement is on individuals, remedies must be available to and used by rights-holders when their rights are denied.78 In this context, any state action that could be characterized as a legal violation (when creationism is presented favorably in any state, or when evolution is not taught in states where state standards require such instruction) creates an everyday legal reality that falls short of law’s strict meaning.79

Lawsuits regarding the teaching of evolution, creationism, and intelligent design-or even public disputes hinting at litigation regarding these matters-are the most obvious way to enforce students’ relevant constitutional rights,80 yet litigation about these matters is so rare that nearly all such filings gain regional prominence, with many generating news coverage on the national level.81 Thus, this Article asks, among other questions, what sort of relationship exists between the political climate or regional location of a state, on one hand, and compliance or lack of compliance with constitutional law requirements and state administrative regulations, on the other.

1. States’ Dominant Political Climates

The terms “Red state” and “Blue state” have become part of American vernacular over the past decade. These terms indicate not just the political party of the Electoral College winner in a given state (Republican and Democrat, respectively), but they also suggest a deeper cultural divide framed by disputes about issues often seen as “moral” or “values”-based issues, including instruction about evolution, creationism, and intelligent design in public school science classrooms.82 For example, emphasizing the recent partisan politicization of scientific issues,83 journalist Chris Mooney titled his recent book, which focuses on science funding and policy during Bush Administration, The Republican War on Science.** The newly-released paperback edition of Mooney’s book devotes an entire chapter to the intelligent design battles: ‘”Creation Science’ 2.O.”85 To a degree, the Red state-Blue state divide is geographical, but certainly not entirely. Thus, the political climate variable by which the data are disaggregated in Part IV is not conceptually duplicative of the geographic region variable.86

2. States ‘ Regional Locations

Considering the degree to which states’ regional locations are correlated with the frequency and manner of evolution, creationism, and intelligent design instruction is another way of evaluating more general cultural differences between states that could influence teachers’ or other educators’ willingness to abide by or disregard constitutional law or state standards.

One way to describe these regional differences is to examine trends in beliefs in evolution, creationism, and intelligent design. According to a recent Pew Research Center survey, nearly 60% of those in the East and the West ascribe to evolution, compared to 45% in the Midwest and only 38% in the South.87 Furthermore, 51% of those in the South ascribe to creationism, compared to 42% in the Midwest, 36% in the West, and 32% in the East.88

This is roughly parallel to differences in religious belief and affiliation from region to region. In the Midwest and South, slightly over 70% of individuals have an “absolutely certain” belief in God, compared to around 60% in the East and West.89 Regional denominational differences may be more culturally descriptive, though. The population of many Southern states is nearly 40% Baptist, with less than 10% Catholic. Catholics comprise a much higher percentage of the population in the Northeast (44% in Massachusetts and 24% even in Maine), and in the West (32% in California and 22% in Montana).90 In the West, the percentage of respondents indicating “no religion” nears 20%, while it hovers slightly above 15% in the Northeast, and ranges from around 10%15% in the South and the Midwest.91 Finally, mainstream Protestants are represented most strongly in the Midwest (with Lutherans alone comprising 22% of Wisconsin residents, for example), although significant differences regarding evolution and other issues divide mainstream and evangelical Protestants.92

III. SURVEY METHODOLOGY AND GENERAL RESULTS

Having established an understanding of the general legal and social contexts in which evolution, creationism, and intelligent design instruction occur in public high school science classes, this section turns to methodological questions. In particular, it explains the survey’s methodology, addresses objections, and sets forth the demographic profile as well as the composite responses of all recent public high school graduates. Readers are reminded of the limitations of the data and of the overand under-representative nature of the nonprobability sample; and, readers are cautioned against generalizing these results too broadly. The survey instrument and protocol were approved by the Drake University Institutional Review Board and the survey was conducted in compliance with IRB procedures.93

A. Methodology of the Survey

A nonrandom sample of students enrolled at eight different public universities in spring 2006 received the survey, with 1053 students completing some portion of the survey, including 972 respondents who completed the entire survey.94 (Responses from incomplete surveys are not reported in any portion of this Article.) The survey instrument was prepared with the input of scholars from various disciplines and piloted twice. Designing the study involved many methodological choices and raised several challenges. This section discusses each of these issues in turn.

1. Selection of Eight States for Comparative Purposes

The study was designed so that the survey responses of recent public high school graduates could be compared in states (1) with strong and weak evolution-related state standards (two groups of four states each), (2) with dominant Red or Blue political climates (two groups of four states each), and (3) that are in four distinct geographic regions of the country (four groups of two states each). Nearly all of these subgroups are internally balanced along the other two factors identified here.95

First, a December 2005 report of the Fordham Institute, a nonprofit education policy think tank, was used to classify states as having “strong” or “weak” evolution-related educational standards.96 The report contains a comprehensive evaluation of the state science standards from the 49 states that have them as well as the District of Columbia and was the second such report issued by the Fordham Institute.97 In the 2005 report, 20 states’ evolution- related standards received a high passing grade of A or B; 17 states, a low passing grade of C or D; and 13 states, a failing grade.98 For purposes of this study, the standards evaluated in the 2005 report needed to be in place by 2001 so that respondents who graduated from high school in 2004 or 2005 would have been in biology class at the time those standards were in effect. Of the eight states selected, six have state standards that were adopted prior to or during 2000 and two have standards that were adopted in 2001 (one of the states augmented its science standards slightly in 2004, but the addition did not materially affect the grade for evolution-related standards assessed by the Fordham Institute).99

Second, states were classified as “Red” or “Blue” if their Electoral College votes went either to the Republican or to the Democrat presidential nominee, respectively, in both of the past two presidential election cycles.100 Electoral College selection was used as a proxy for political climate rather than the party of the governor or other elected state- or federal-officeholders because although significant variation can exist among members of the same party within and between states,101 the presidential candidates remain constant across all 50 states. The eight target states can be divided into two groups of four Red and four Blue states which are balanced by the strength of states’ evolution-related standards: of the Red states, Indiana and Tennessee had strong science standards and Arkansas and Montana had failing science standards.102 Of the Blue states, California and Massachusetts had strong science standards and Maine and Wisconsin had failing science standards.103 Given the many constraints for selecting target states, it was unfortunately not possible for the two groups to be entirely balanced geographically. The Red states group contains two states from the South, one from the Midwest, one from the West, and none from the Northeast; conversely, the Blue states group contains two from the Northeast, one from the Midwest, one from the West, and none from the South.104 Third, the four geographic regions of “Northeast,”"South,”"Midwest,” and “West” are the regions recognized by the U.S. Census Bureau.105 As discussed immediately above, among the eight states in the sample, the Midwest and West geographic regions contain internal Red state-Blue state variation, but the Northeast and South regions do not.106 However, each regional pair of states contains one state with strong evolution- related standards and one state with weak evolution-related standards.

Therefore, the eight states selected for this survey have the following characteristics:

Because California public high school students comprised nearly 60% of all public high school students enrolled in the eight selected states during the 2004-2005 school year (though only 29% of the recent public high school graduates in this survey sample), weighting the data would have produced statistics overwhelmingly skewed toward California; thus, data presented throughout this Article are in a nonweighted form.107

2. Selection of the Sample

The tradeoffs of using a nonrandom sample are presented later at Part II.B.1. as part of the discussion of potential methodological objections.108 This section discusses the identification of the sample pool, once the nonrandom selection method had been determined. The sample pool was selected so that students solicited to participate in the survey were likely to have attended high school in communities distributed throughout a given state between the years 2000 and 2005.

First, surveying students in college rather than in high school ensured that nearly all respondents were at least 18 years old. At the time of the survey, respondents were enrolled in college and even the youngest students were about to complete their first year.109 By contrast, high school students-almost all under age 18- only could participate in a survey such as this with signed parental permission,110 which would restrict the number of participants significantly and also introduce additional bias as discussed below. Furthermore, obtaining a sample of students from high schools across the state would be much more difficult.111

Second, public high schools often teach evolution (and thus creationism and intelligent design) towards the end of tenth grade, although state standards do not necessarily require this timing.112 Several states’ science standards include general principles of evolution as early as elementary school113 and other principles through the senior year of high school.114 As a result, conducting a survey of college students in eight different states, all of whom have graduated from high school, ensured that respondents had been exposed to the full measure of whatever evolution, creationism, or intelligent design instruction they were going to receive in high school.

Third, although the survey was distributed in a manner akin to the distribution of a paper survey, it was conducted online through a secure website; in other words, respondents could only access the survey if provided with the hyperlink and would not find the survey if they were merely surfing the web.115 Conducting the survey online eliminated data entry errors, eased data collection, and also made it possible to require that respondents answer each question before proceeding to the next. (This last advantage is not possible in a paper survey, and is the reason the statistically serious problem of item nonresponse error occurs, requiring missing values to be estimated.116) Additionally, the time required to complete a lengthy survey is significantly less when the survey is online as opposed to being read to the respondent, as is the case with a telephone survey. It appears unlikely that any respondents completed the survey more than once, although this cannot be determined definitively.117

Fourth, the use of an online survey tool was connected to the decision to survey college-age students rather than high school students. Although approximately 75% of homes with a phone line have internet access,118 home internet access is highly correlated with family income level.119 Although all elementary and secondary schools in the United States have some degree of internet access,120 teachers likely could be reluctant to allow students to use a portion of the limited time during the school day to complete this survey; and, even a full classroom set of computers may only be available to students in an elective computer class or during study hall. Furthermore, verifying parental consent would present additional challenges. By contrast, because computer use and internet access are common features of college and university campus life,121 students regularly have access to computer labs if they do not own their own computers122 and would not need to take time away from class to participate in the online survey, therefore potentially reducing the reluctance of faculty to distribute the survey solicitation to their students.

Fifth, because respondents are university students and college and university students as a group are more likely to come from families with a higher level of wealth, the respondent group has some amount of inherent class bias, which also is likely related to the educational attainment of a respondent’s parent(s).123 Awareness of this factor influenced the decision to survey students at public rather than private universities. For the 2004-2005 school year, the average amount of undergraduate tuition, fees, room, and board at a four-year public university was $11,441, compared to $26,489 at a private university.124 As a family’s average annual income increases, so, too, does the likelihood that children from that family will attend a private university.125 Thus, four-year public universities are somewhat more reflective of the socioeconomic diversity of the country than four-year private universities.

In sum, by surveying students enrolled at public universities about their experiences in high school science class, a sample pool of respondents was identified who did not require parental permission to participate because they were nearly all at least 18 years old; who had ready access to the internet thus giving them the ability to complete the survey; most of whom who had attended public high schools in the same state as they attended college and had done so during a given timeframe; and, who represented a greater socioeconomic variation than students at private colleges and universities. To maximize the utility of the sample, students were surveyed at one public university in each of the eight selected states that best satisfied three criteria:

(1) The university has a large undergraduate enrollment, ranging from just over 8,000 at two schools126 to approximately 12,000- 15,000 at three schools,127 to 19,000-23,000 at the final three schools.128 To avoid a sample self-selected simply because of respondents’ investment in the topic, faculty were solicited at each school to distribute the survey solicitation by forwarding an email message to students enrolled in a specific class or to post such a message on the online class bulletin board. The message contained a brief solicitation, describing the survey as “about your high school science education” and a hyperlink to the survey which was necessary to access the survey.129 To monitor any pro-evolution bias of students enrolled in introductory biology (one of the classes through which the survey was distributed), the survey also was distributed to students in an introductory psychology or sociology class at participating universities.130

(2) The college or university has a high in-state enrollment and instate students’ hometowns’ are not disproportionately concentrated in one area of the state. Considering the first of these two factors increases the likelihood that students in a given class will have attended high school in that state, and the latter minimizes within- state regional bias.131

(3) The college or university has a small non-traditional student enrollment, ideally less than 10% and definitively not more than 20%.132 To effectively consider student responses in light of state science standards it is necessary to ensure that a particular set of standards was in place when most respondents were enrolled in high school. Students who were first- and second-year college students during spring 2006, when this survey was conducted, would have graduated from high school in 2005 or 2004, unless they were non- traditional students who did not attend college immediately after graduating from high school.

3. Developing the Survey Instrument

The survey instrument benefited from the input of faculty across the academy with expertise in biology, political science, sociology, and of course law. Furthermore, it was piloted initially with a small group of undergraduates, and then with a larger group of law students; revisions were made based on the input of both groups of students.

The survey draws on questions asked in one survey of University of Minnesota students,133 surveys targeted to high school science teachers,134 newly-created questions which correlate to Establishment Clause case law (this last set effectively asks whether students view their experiences as having satisfied various constitutional tests and will be the subject of subsequent work), standard demographic questions,135 and questions inquiring about respondents’ personal views about evolution, creationism, and intelligent design that are similar to questions included on national opinion polls.136 The survey contained 58 questions,137 although due to skip logic,138 a respondent would be presented with, and required to answer before moving to the next screen, a total of only 31 to 48 questions. Respondents repeatedly were assured their responses would be confidential.139 B. Potential Methodological Objections

While no survey is without methodological challenges, the survey design attempts to neutralize resulting bias to the extent feasible and this presentation seeks to acknowledge remaining bias.

1. Sampling Method

In this survey, a targeted group of college students rather than a probability sample of 18-20 year olds was surveyed about their high school science education. This method of nonrandom sampling, generally known as “convenience sampling,”140 is not uncommon in social or behavioral sciences141 and, on occasion, has been recognized as “reliable and fruitful” in terms of contacting respondents who otherwise are difficult to locate.142 Nonrandom sampling falls short of achieving the gold standard of validity of probability sampling, of which the simple random sample is the best known example.143 However, when the researcher is aware of how the sample population compares to the population as a whole, a convenience sample still can yield results that form a valuable starting point for further discussion and research.144 As this section will make clear, although the likelihood of any nonrandom sample being representative of the population is very small, the survey protocol here was designed so that the survey would not intentionally aggravate the unrepresentativeness of the population with the exception of selecting all university students. Furthermore, the demographic profile of all recent public high school graduate respondents is presented in Part ILCl. for the purpose of quantifying the overrepresentation and underrepresentation of various groups whose answers are analyzed in this Article.

A convenience sample was employed in this situation for several reasons: first, the survey was too lengthy to add to another survey already being distributed to this demographic group via random sampling. Second, a random sample survey was cost-prohibitive.145 Third, individuals in the targeted demographic (age 18-20) are notoriously difficult for even professional pollsters to contact: nearly 20% of 18-25 year olds use a cell phone but do not live in a home with a land line,146 compared to an estimated 7-9% of the American population.147 (Pollsters typically contact individuals in a random sample poll via use of a computer-generated number which is connected to an auto-dialer, and it is a violation of federal law to place calls to a cell phone using an auto-dialer.148) Thus, even a random sample of 18-20 year olds generated in customary fashion would be unlikely to have the traditional reliability we expect from a standard random sample.149

Finally, because the sampling method was nonrandom, a margin of error cannot be calculated.150 It should be noted that in a random sample, a larger sample size reduces the margin of error, and the same concept also is generally applicable in nonrandom sampling.151

2. Nonresponse Bias and Self-Selected Response Bias

Participation in the survey was voluntary. A classic pitfall of voluntary surveys is that those who are most likely to respond are those with particularly strong views about the subject of the survey,152 thus skewing the data collected.153 In the specific context of surveying college students about evolution, creationism, and intelligent design, at least two researchers have speculated (and with good reason) that students with creationist-oriented beliefs refrained from participating in data collection regarding these issues.154 That may be the case in this survey, as well, where the percentage of respondents who expressed a belief in evolution, and not in creationism or intelligent design, was significantly higher than the national average.155

As a general matter, nonresponse bias is reduced to the extent that the sample is effectively representative of the population surveyed.156 For this reason, the demographic profile of the survey respondent group is presented in the following section, compared to the population of United States adults and, when possible, to a subset of the population closer to the targeted respondents in age.

In an attempt to mitigate the effects of nonresponse bias and selfselected respondent bias by increasing response among potential respondents who may not feel strongly about the issues raised and thus be less likely to participate or complete such a survey, the survey solicitation described the survey as only “about your high school science education.” The solicitation message also informed students that one respondent would win a new video iPod and four respondents would receive one $25 iTunes gift card each.157

Of all students who received the solicitation message, 1045 completed at least some portion of the survey for an overall response rate158 of 28.78%, although response rates varied substantially from school to school (8 students from other institutions also participated in the survey, for a total respondent pool of 1053).159

3. Recall Error

To answer questions about the instruction they received in a high school science class about evolution, creationism, or intelligent design, recent high school graduate respondents in most cases are relying on a three- or four-year old memory of their educational experiences. This delay, admittedly, is problematic: students may not remember which topics were covered in their high school science classes and in what level of detail, or their memories may have changed somewhat over time.160 Although it is not possible to counteract or quantify these aspects of recall error, it should be noted that the uniqueness of instruction about evolution and related topics may cause memories about such instruction to be recalled with greater ease-a concept termed “salience” when studied in the field of cognitive psychology.1’1 Furthermore, because there are few imaginable similar events (excepting a college biology class) that would occur between the time of high school science class instruction and the time of the survey for 2004 and 2005 high school graduates, the chance of new events causing interference with respondents’ original memories appears low.162 Finally, so as to not force recall speculation, respondents always were provided with the option “Don’t Remember” when the question asked for their memory of past events.163

4. Textbook Content

Although not a methodological challenge in the traditional sense, the inability to account for the content of a textbook used in a particular respondent’s high school biology class also may impact the results of this study. Textbook content forms the basis for a very high percentage of classroom instruction and assignments in states across the country (presumably in strong standards and weak standards states alike).1*4 Approximately 70% of public school textbooks are produced by only four major publishers; strong standards states and weak standards states draw from the same, albeit somewhat varied, pool of texts.16S

Educational standards are not identical in any two states,166 but a small number of populous states that purchase significant numbers of textbooks have long had a significant influence on textbook content. Purchases from California, Texas, and Florida alone make up over 30% of the annual $4.3 billion textbook industry.167 Thus, an interesting side-effect of variations in state standards is some variation in texts-for example, a text marketed for adoption in large strong standards states such as California is rather different in terms of its evolution coverage than one for which the market plan is adoption in large weak standards states such as Florida and Texas.168 And, it appears that classroom teachers are not the only ones responding to community pressure to avoid introducing evolution or exploring it in depth-so, too, are major textbook publishers, driven by the desire to have their textbooks adopted169 and exempt from the requirement that the texts be subject to peer review.170 As a result, even a book intended to be marketed in California may have less evolution-related content than the California standards proscribe if the publisher wants the book to be more generally marketable; however, it still may have more evolution-related content than a book targeted for weak-standards states. Perhaps not surprisingly, the content of these textbooks is criticized by many across the political spectrum.171

C. General Results

This section first provides an overview of the demographic characteristics of the nearly 600 recent public high school graduates, and then presents a summary of their answers regarding the manner and frequency of evolution, creationism, and intelligent design instruction.172 The information provided by this subgroup (recent public high school graduates) was incredibly similar to the set of answers provided by all respondents, with the exception of two slight trends: earlier high school graduates were somewhat more likely to indicate “do not remember” as an answer to various questions and somewhat less likely to select “strongly agree” or “strongly disagree” (or equivalent answers representing the ends of the spectrum).173

1. Demographic Overview of Respondents

The survey included 21 questions that can be labeled, generally, as demographic questions.174 The demographic profile of the 573 respondents who constitute the group “recent public high school graduates” is as follows (where the demographic profile of this subgroup differs materially from the demographic profile of all respondents, such difference is noted).

a. “Recent Public High School Graduates”

Of all 972 respondents who completed the entire survey, 94.86% attended all four years of high school in one state.175 85.03% attend college in the same state they attended high school.176 87.74% attended a public high school all four years.177 56.07% graduated from high school in 2005, followed by 22.84% who were 2004 high school graduates.178 Combining these characteristics, nearly 60% of all respondents (573) attended a public high school for all four years in one of the eight states designated as the subject of this study and graduated in 2004 or 2005.179 These 573 respondents comprise the subgroup referred to throughout this Article as “recent public high school graduate respondents.” b. General Demographic Characteristics

This section presents the demographic profile of the recent public high school graduate respondents (“the respondent group”) with regard to race/ethnicity, sex, public/private high school attendance, respondents’ perceptions of their family’s socioeconomic status while they were in high school, respondents’ majors, and the college class through which respondents received the survey.

First, race and ethnicity. Compared to all 18-24 years olds in the United States, Asian-Americans are overrepresented in the respondent group by about 14% and African-Americans and Latinos are underrepresented (by 12% and 16%, respectively).180 Whites are underrepresented by about 4% compared to the 18-24 year old United States population.181 A recent study by the Pew Research Center suggests that, among all U.S. adults, African-Americans ascribe to creationism approximately 10% more frequently than do Whites or Latinos, and to evolution 10% less frequently.182 Religious affiliation is discussed at a later point.

Second, sex. Women were overrepresented in the respondent group by 18% compared to the general population age 18-24 (66% to 48%, respectively), but are only overrepresented by 9% compared to their enrollment in colleges and universities.183 In the adult U.S. population, women are less likely than men to favor evolution as an explanation for the origin and diversity of the species, with 43% of women and 54% of men declaring a belief in evolution, and 47% of women and 36% of men claiming a belief in creationism in a recent Pew Research Center study.184

Third, public versus private school enrollment. The percentage of all survey respondents who were home schooled and who attended private secondary schools reflect the same percentages as the national high school enrollment statistics. However, respondents who attended a private high school for all or part of those four years, or who were home schooled, are not included in the data reported in this Article, because those respondents are, by definition, not recent public high school graduates.185

Fourth, socioeconomic status. Although the United States government has designated an income line below which an individual or family is said to be living in poverty, no other such lines are drawn with regard to socioeconomic status. It is assumed that students are unlikely to know their family’s annual income, and that socioeconomic self-classification tells much more about self- perception than about membership in a particular income group.186 Compared to the national population, respondents are underrepresentative of individuals living in poverty by more than 10%. As well, slightly more than 80% of respondents perceived their families as middle-class or upper middle-class.187 The Pew Research Center has documented a positive correlation between income and belief in evolution, and, similarly, a negative correlation between income and belief in creationism.188

Fifth, students’ majors. Compared to the fields in which United States bachelors degrees were awarded in the 2003-2004 academic year, science majors are overrepresented in the respondent group by approximately five-fold (comprising approximately half the respondent group), and business majors are underrepresented by about 15%; soft science majors, social science majors, and education majors are represented in amounts roughly equivalent to the national statistics for majors at graduation.189

Sixth, survey distribution method. Respondents who received the survey through an introductory biology class were overrepresented by about 2:1, compared to those who received it through an introductory psychology or sociology class.190

c. Educational Level

The survey pool consisted entirely of students enrolled at four- year colleges and universities. This leads to two possible sources of education-level bias: the direct bias of the students’ own education level, and the indirect bias of their parents’ educational level.

In 2005, of the entire U.S. population over age 25,27.6% had received a bachelor’s degree; of the 25-29 age group, 28.6% had received a bachelor’s degree.191 Respondents’ parents thus are overrepresentative of college graduates in the general U.S. adult population by a 2:1 margin.192 Respondents are vastly overrepresentative of this group, with less than 0.5% anticipating they would not receive a bachelor’s degree, an additional 3% uncertain about what level of education they will attain, and the remaining 97% anticipating that they will receive a bachelor’s degree, if not a graduate degree.193

According to a recent Harris Poll, 17% of U.S. adults with a high school degree or less formal education ascribe to the theory of evolution, with belief in evolution rising as the level of education increases to 35% of U.S. adults who have a postgraduate degree- interestingly, U.S. parents’ weekly attendance at religious services also rises as their level of formal education increases, from 28% of parents who are not high school graduates to 43% of parents with a college degree.194 However, the level of education and belief in creationism are inversely proportional, with 73% of U.S. adults with a high school degree or less ascribing to creationism, and this percentage declining to 42% of those with a postgraduate degree.195 Belief in intelligent design, like evolution, is directly proportional to level of education, increasing from 6% of U.S. adults with a high school degree or less to 17% of those with a postgraduate degree.196 These findings by the Harris Poll are roughly consistent with those presented in a recent report by the Pew Research Cent