• E-mail
• Print
• Comment
• Font Size
• Digg
• del.icio.us
• Discuss article

Impact of a High School Graduation Examination on Social Studies Teachers' Instructional Practices

Posted on: Friday, 28 October 2005, 03:01 CDT

By Vogler, Kenneth E

Abstract

The purpose of this study was to explore the impact of high- stakes tests on teachers' instructional practices. Data were obtained from a survey instrument given to a stratified random sample of Mississippi social studies teachers who teach the same content that is tested on their state's high school graduation examination. An analysis found teachers spending the most time preparing students for the examination were more likely to use traditional, teacher-centered practices such as textbooks, multiple- choice questions, lecturing, and textbook-based assignments. Also, teachers' use of instructional practices was most influenced by factors relating to sanctions attached to the examination rather than items such as personal desire or belief about using "best" practices.

Introduction

I use the entire academic year preparing my students for the United States history subject area exam. My choice of instructional delivery and materials is completely dependent on preparation for this test. Therefore, I do not use current events, long-term projects, or creative group/corporate work because this is not tested and the delivery format is not used. All my tests reflect the testing format of the subject area tests- multiple-choice and open ended questions. This delivery and curriculum format is not used in my other classes (United State Government and Economics). In these non-tested classes, I do use a variety of instructional methods and curriculum resources. While I agree with the principle of student / teacher / administrator accountability, but by making the goal of my United States history course the ability to pass the state test, I'm afraid that all meaningfulness and relevancy to history is being lost on my students. As a result, they have a better factual base but a worse conceptual understanding of the subject and what it is good for.

-A Mississippi High School United States History Teacher

The high school history teacher's comments echo a frustration felt by many teachers who teach the same content that is tested on their state's high school graduation examination: What instructional practices should I use? On the one hand, these teachers want to use instructional practices that make their classes interesting, develop students' higher-level thinking skills, and foster a passion or at least spark students' interest in the subject and why it is relevant. On the other hand, these teachers have the immense responsibility to prepare their students for the examination. Failure to do so may put in jeopardy their students' ability to earn a high school diploma and their regard as "effective" teachers.

The impact of high-stakes tests such as a high school graduation examination on teachers' instructional practices is a very relevant concern, with, as of yet, no clear consensus as to what the impact is. Although researchers such as Barksdale-Ladd and Thomas (2000), Jones and Johnston (2002), McNeil (2000), Vogler (2002), and Yarbrough (1999) have found that teachers changed their instructional practices in response to high-stakes testing, there is still considerable ambiguity about the nature and intensity of this relationship (Firestone et al., 2002; Grant, 2001). Factors such as subject and grade level taught, personal beliefs, type of high- stakes assessment, and professional development all have the potential to impact this relationship in varying degrees (see Cimbricz, 2002; Jones, Jones, & Hargrove, 2003).

The purpose of this study is to explore further the impact of high-stakes tests on teachers' instructional practices. More specifically, this study focuses on the instructional practices used and factors influencing their use by Mississippi social studies teachers who teach the same content that is tested on their state's high school graduation examination. I used a state-wide teacher survey to answer the following questions:

Central question:

In what manner does a high school graduation examination influence instructional practices?

Additional questions include the following:

1. What instructional practices do teachers use?

2. How often do teachers use these instructional practices?

3. What factors have influenced their use?

I begin with a brief review of testing and accountability systems and information about Mississippi's high school graduation examination. Then I describe the research method and examine the results of the study's central question and three additional questions.

Testing Policies and Accountability Systems

Accountability, high standards, and curriculum alignment are just a few of the terms used in conjunction with the newest method to reform public schools-testing. The reauthorization of the Elementary and secondary Education Act (1965) and its instantiation in No Child Left Behind (2002) is an example of the tremendous confidence policymakers such as the President of the United States and members of Congress have in testing and assessment systems to improve public education.

According to Natriello and Pallas (1999), there are three very compelling reasons for policymakers to justify the use of statewide testing and accountability systems. First, testing has the potential to influence the behavior of students, teachers, and administrators. For students, individual test results can help pinpoint areas in which they need additional instruction. For teachers and administrators, test results allow the public to scrutinize and compare the achievement of students under their charge with student achievement in other public schools. second, attaching stakes such as high school graduation to the results of testing and accountability systems assures the public of a basic level of competency for high school graduates. Finally, testing has an inherent efficiency and scientific legitimacy as a performance monitoring process. The ratio of information gathered to expenses incurred as well as the ability to assess the validity and reliability of scores generated by standardized measures seem to make testing and accountability systems a very inexpensive investment.

The linchpin of testing and accountability systems is presumed to be the use of stakes attached to test scores as a way to exert significant influence on classroom learning and instructional practices. These stakes have included incentives such as cash awards to schools or individual teachers who demonstrate high levels of student performance. They also have meant consequences for schools, individual teachers, and students; these consequences include publishing test results, delaying grade-to-grade promotion, possible takeover of schools that continue to demonstrate low levels of student performance, and putting off high school graduation. Currently, 20 states require students to pass a statewide exit or end of course examination to receive a high school diploma. But of these 20 states, only Georgia, Louisiana, Mississippi, New Mexico, and New York include passing an examination on social studies as a high school graduation requirement (Skinner & Staresina, 2004).

Mississippi's High School Graduation Examination

In 2000, under State Board of Education Policy IHF-1, the Subject Area Testing Program (SATP) became a requirement for high school graduation (Mississippi Department of Education, 2004a). This test program was designed to, among other things, evaluate the performance of Mississippi schools and districts in teaching the Mississippi Curriculum Framework (see Appendix A) (Marchette, 2003). The SATP consists of end-of-course, criterion-referenced tests in Algebra I, Biology I, United States History from 1877-Present, and English with a writing component (Mississippi Department of Education, 2004b). In 2002, after a standard setting and a phase-in process, the social studies portion of the SATP was completely implemented (see Appendix B) (Mississippi Department of Education, 2004c). Students now must pass the United States History from 1877- Present SATP examination to graduate high school.

Method

A survey instrument (see Appendix C) was used to answer the research questions. Part I of the survey instrument contains items pertaining to instructional practices used and the extent to which they are used: Part II contains items pertaining to factors influencing instructional practices used: and Part III contains items pertaining to demographic information. Also, a section is included asking if and how much instructional time is spent preparing students for the high school graduation examination. Finally, there is a section called "Comments" which offers respondents an opportunity to talk about the instructional practices they use to prepare students for the high school graduation examination.

Sample Selection

This researcher created a stratified random sample of high school social studies teachers using geographic region and past student success on the SATP. First, school systems were grouped according to geographic region: East, Middle, and West Mississippi. second, the school systems in each region were ranked according to student success on the latest (2002-2003) SATP. Quartiles were generated using this ranking. At least four, but no more than six school systems from each quartile participated in the study.

A total of 55 school systems agreedto participate in the study. All high school social studies teachers teaching United States History 1877-Present from each participating school system were given the survey instrument by their principals. The content covered in this United States history course, according to the Mississippi State Framework, is the same social studies content tested on the SATP. One hundred and seven (107) teachers, or 63.7% of the total population surveyed, completed the survey instrument.

Data Analysis

Analysis of the data began by computing frequency and means on questions in the survey instrument. This was followed by conducting a number of chi square analyses in an attempt to uncover relationships among variables. Finally, responses in the "Comments" section were analyzed and grouped according to content expressed.

Comparison of Survey Sample and State Teaching Population

The researcher compared survey respondents with the state's teaching population using data obtained from Part III of the survey instrument and the Mississippi Department of Education. Table 1 is a comparison of the frequency distribution between the survey response sample and the Mississippi high school United States history teacher population for gender. (see Table 1 at the end of this text.) The demographic variable teaching experience was compared in Table 2. (See Table 2 at the end of this text.) Table 3 is a comparison of the frequency distribution between the survey response sample and the Mississippi high school United States history teacher population for highest education level obtained. (See Table 3 at the end of this text.)

With the exception of a slightly lower percentage of teachers with 0-6 years of experience, Tables 1-3 show that participants in this study are representative of the Mississippi high school United States history teaching population in terms of gender, teaching experience, and highest level of education attained.

Results

This section begins with a preview of the study's most interesting results. First, teachers report that they most often use traditional, teacher-centered practices such as textbooks, multiple- choice questions, lecturing, and textbook-based assignments. Also, teachers report they least use instructional practices such as role playing, project-based assignments, interdisciplinary instruction, and problem-solving activities. Second, 89 teachers, or 83.2% of the total sample, acknowledged spending class time preparing students for the high school graduation examination. Teachers spending over two months preparing students for the graduation examination are more likely to use more textbooks, multiple-choice questions, visual aids, textbook-based assignments, rubrics or scoring guides, and worksheets and less likely to use creative/critical thinking questions, project-based assignments, group projects, true-false questions, and role playing than teachers spending one day to two months preparing students for the examination. Teachers spending one day to two months preparing students for the graduation examination typically use more open-response questions and creative/critical thinking questions, and fewer lessons-based on current events, cooperative learning/group work, newspaper/magazines, computers/ internet and education software, and interdisciplinary instruction than the other group. Third, teachers overwhelmingly felt that the instructional practices they use are mostly influenced by sanctions attached to the high-stakes test scores rather than factors such as personal beliefs. An "interest in helping my students attain test scores that will allow them to graduate high school" and an "interest in helping my school improve high school graduation scores" were the most commonly reported influences.

Difference Between Student-Centered and Teacher-Centered Practices

Before discussing the analysis of Part I of the survey instrument, the instructional practices used, I it is best to first have an understanding of the teaching methods understood to be most effective for student learning. Researchers have identified two general methods or approaches to teaching-student-centered and teacher-centered (Airasian & Walsh, 1997; Eggen & Kauchak, 2001).

Student-centered teaching can be thought of as an application of a constructivist theory of student learning. Constructivists believe that students actively construct their knowledge through interacting with their physical and social environments (Piaget, 1973; Vygotsky, 1978), rather than act as empty vessels into which knowledge is poured. The other approach to teaching is called teacher-centered. This approach places the teacher at the center of all activities during instruction (Jones, Jones, & Hargrove, 2003). Typically, this method of instruction includes a preponderance of practices such as lecture, lecture and discussion, and direct instruction (Eggen & Kauchak, 2001).

Guiding questions: 1. What instructional practices do teachers use?

2. How often do teachers use these instructional practices?

Part I of the survey instrument was designed to answer the study's first two guiding questions. An analysis begins with an examination of frequency tables and the mean response for each item. The larger the mean of an item, the more respondents used the particular instructional practice or tool. Table 4 represents those practices respondents reported using regularly or often. Table 5 represents those practices that teachers reported using less often or not at all.

By implication, the data in Tables 4 and 5 provide a glimpse into which teaching approach, student-centered or teacher-centered, is more often used by Mississippi high school United States history teachers. (See Table 4 and 5 at the end of this text) An analysis of the data supports the conclusion that the survey respondents largely favor more traditional, teacher-centered approaches over student- centered practices. First, teachers (94.4%) reported that textbooks were, by far, the most commonly used instructional practice. This was followed by instructional practices or tools such as multiple- choice questions (88.8%), visual aids (83.2%), supplementary materials (83.2%), lecturing (82.3%), and textbook -based assignments (78.5%). These practices and tools are all instruction typical of a teacher-centered approach. In fact, of the first seven instructional practices or tools respondents reported using the most, only open-response questions (84.1%) can be considered instruction more in line with a student-centered approach. Second, respondents reported spending the least amount of instructional time using response journals (90.6%), role playing (87.8%), true-false question (86%). group projects (83.2%), project-based assignments (73.8%), computers/educational software (72.6%), interdisciplinary instruction (70.1%), and problem-solving activities (68.5%). With the exception of true-false questions, the instructional practices or tools used least by respondents are student-centered approaches.

Minimal Demographic Differences in Instructional Preferences

Next, a number of cross tabulations and chi square analyses were conducted to determine if any significant differences in the instructional practices used or not used among demographic categories listed in Part III of the survey instrument. Table 6 represents statistically significant instructional preferences divided along gender lines. Table 7 represents statistically significant practices broken down by years of experience. Categories were "collapsed" to ensure numbers sufficient to meet the statistical requirements for a chi square analysis. (See Table 6 and 7 at the end of this text)

As evident in these analyses, although there were some statistically significant differences such as a greater percentage of females regularly or mostly used project-based assignments, charts/webs/outlines, and computers/internet than males and a greater percentage of teachers with 7-14 years of experience regularly or mostly used open-response questions than teachers with 15-24 years of experience, the effect was minimal. These exceptions aside, there were no glaring dissimilarities between the instructional practices or tools used or not used by any demographic group or by the response population as a whole.

Preparing Students for the High School Graduation Examination

Questions #31 and #32 in the survey instrument ask about preparing students for the high school graduation examination (See Appendix C). Eighty-nine respondents, or 83.2% of the total sample, acknowledged spending instructional time preparing students for the high school graduation examination. After "collapsing" the preparation time categories into "1 day to 2 months" and "over 2 months," cross tabulations chi square analyses were conducted to determine if there were any significant differences between the instructional practices used or not used and the "collapsed" preparation time categories. Table 8 is a comparison of the instructional practices and tools mostly used by respondents spending instructional time preparing students for the high school graduation examination. (See Table 8 at the end of this text.) Table 9 is a comparison of the instructional practices and tools least used by survey respondents spending instructional time preparing students for the high school graduation examination. (See Table 9 at the end of this text.)

Table 8 shows that teachers preparing their students for the high school graduation examination are using instructional practices and tools more in line with a teacher-centered learning approach rather than a student-centered learning approach. In fact, as shown in table 8, of the six instructional practices used most often, five are teacher-centered. And, as evidenced in Table 9, all 15 of the instructional practices and tools used the least by both groups of teachers are in line with a student-centered learning approach.

The researcher wondered w\hat instructional practices were being used by teachers who didn't use any instructional time preparing their students for the high school graduation examination. Because there were only 18 respondents who didn't spend any instructional time preparing their students for the high school graduation examination, it was decided to include respondents who used the least amount of instructional time preparing their student for the high school graduation examination (1 day to 1 week) in these calculations. The addition of these respondents brought the total to 30 teachers or 28% of the sample. Unfortunately, no clear pattern emerged. These teachers were using all the instructional strategies and tools listed in the survey instrument in varying degrees.

Guiding question 3. What factors have influenced their use?

Part II of the survey instrument was designed to answer the study's third guiding question-the factors influencing the instructional practices and tools respondents are using (See Appendix C). In order to provide descriptive information, analysis of this date also begins with an examination of frequency tables. The frequency tables also provide the mean, standard deviation, and standard error for each item. Table 10 shows the influence factors based upon the mean in descending order from highest to lowest value. (Any mean over "3.00" would indicate some perceived amount of influence.) (See Table 10 at the end of this text.)

A cursory examination of Table 10 reveals that 96.3% of respondents agreed that an "interest in helping my students attain test scores that will allow them to graduate high school," and 94.4% agreed that an "interest in helping my school improve high school graduation examination scores" had an influence on instructional practices they use. These factors were followed by the "format of the high school graduation examination" (83.1%), "personal desire" (83.1%), and "belief these are the best instructional practices" (80.3%). The factors least influencing teachers' use of instructional practices and tools were "interactions with parents," (40.2%) and an "interest in obtaining a monetary award for my school" (15.1 %).

Comparisons were made between influence factors and respondents spending over 2 months and respondents spending 1 day to 2 months of instructional time preparing their students for the high school graduation examination. Table 11 shows the results of these comparisons. (See Table 11 at the end of this text.)

In this table, one can clearly see the impact the stakes attached (i.e., high school graduation) to the examination have had on teachers who spend the most instructional time preparing their students for the high school graduation examination. One hundred percent of respondents spending over two months preparing students for the graduation examination said that an "interest in helping my students attain test scores that will allow them to graduate high school," and an "interest in helping my school improve high school graduation scores" were factors influencing their instructional practices. These two factors also appear to influence respondents spending one day to two months of instructional time preparing students for the high school graduation examination. And, for all but one factor, "interest in avoiding sanctions at my school," the total percentage of agreement was greater for respondents spending over two months of instructional time preparing their students for the high school graduation examination than for their peers.

Comparisons were also made between influence factors and respondents who either didn't use or used relatively little instructional time (1 day to 1 week) preparing their students for the high school graduation examination. The result, the influence these factors had on the instructional practices used by this group of teachers was remarkably similar to groups of respondents spending a lot more instructional time preparing their students for the high school graduation examination. One hundred percent of respondents who either didn't use or used relatively little instructional time preparing their students for the high school graduation examination said that an "interest in helping my students attain test scores that will allow them to graduate high school," and 96.6% agreed that an "interest in helping my school improve high school graduation examination scores" had an influence on instructional practices used. These influence factors were followed by "interaction with colleagues (76.6%),"personal desire" (73%), "belief these are the best instructional practices" (73%), "interest in avoiding sanctions at my school (70%), and "format of the high school graduation examination" (66%). The factors least influencing these teachers' use of instructional practices and tools were "interactions with parents," (46.6%) and an "interest in obtaining a monetary award for my school" (13.3%).

Discussion

The purpose of this study was to explore the impact of high- stakes tests on teachers' instructional practices. More specifically, this study focused on the instructional practices used and factors influencing their use by Mississippi social studies teachers who teach under their state's high school graduation examination. From my analysis, Mississippi teachers are far more likely to use traditional, teacher-centered practices than student- centered practices. But, what does this mean? If the question is which of these approaches, student-centered or teacher-centered, is most effective, the answer is both. Student-centered methods are more effective for teaching complex objectives and developing higher level thinking skills, and teacher-centered methods are more effective for teaching procedural skills and organizing knowledge to review facts and identify relationships (Good & Brophy, 2000). Effective teachers use both methods, depending upon the needs of their students and objectives of each lesson (Airasian & Walsh, 1997; Pressley, Rankin, & Yokor, 1996; Zemelman, Daniels, & Hyde, 1998).

Although it is impossible to describe the perfect balance between student-centered and teacher-centered instruction due to factors such as subject, grade level, and lesson objectives (Jones, Jones, & Hargrove, 2003), research on best practices (Zemelman, Daniels, & Hyde, 1998; Wenglinsky, 2000) and position papers of professional teaching organization(e.g., National Council of Teachers of Mathematics and National Council of Social Studies) have advocated instructional practices more connected to constructivist theory and student-centered methods. Teacher-centered approaches, by contrast, are seen as instruction only useful for developing lower level thinking skills such as identifying, memorizing, and listing information.

According to the data, Mississippi teachers are using instructional practices and tools such as textbooks, multiple- choice questions, lecturing, and textbook-based assignments far more than role playing, project-based assignments, interdisciplinary instruction, and problem-solving activities. These results imply that respondents are using instructional practices and tools more in line with a teacher-centered learning approach. It is this type of instruction that may be perceived as useful for state-level tests, but it reduces history to merely a list of people, places, and dates rather than an opportunity for students to experience history and use it as a bridge to connect all academic areas.

Nevertheless, those state-level tests in Mississippi matter considerably to teachers. Over 80% of the total sample acknowledged spending class time preparing students for the high school graduation examination. Comparisons were made between teachers spending 1 day to 2 months preparing their students for the high school graduation examination and teachers spending over 2 months preparing their students for the high school graduation examination. The result, teachers spending the most amount of instructional time preparing their students for the high school graduation examination are far more likely to use teacher-centered approach than their peers. Presumably, these teachers feel that using teacher-centered instructional practices is the best way to prepare their students for the high school graduation examination.

In regard to the question of what factors influence teachers' instructional decisions, according to the data, well over 90% of respondents believe the state test influences the instructional practices they use. Comparisons between respondents reporting more and less attention to test preparation yielded predictable results: Teachers who spent more than two months on test preparation were far more likely to use teacher-centered practices than those who spent less time on test preparation.

For respondents, the amount of instructional time they spent preparing their students for the high school graduation examination mattered little. Their message was clear and can be summed up in a comment expressed by a female with a bachelor's degree and 25-29 years of experience: "Regardless of how I feel about them, tests and the high-stakes attached to test results matter the most. They affect not only the curriculum taught but the instructional practices used to teach it."

Conclusion

The blueprint to achieve educational accountability is seemingly quite simple. First, design state-wide curriculum for each content area-a "framework" that includes standards and benchmarks for each grade level. Then, to make sure teachers follow and teach this curriculum, mandate statewide tests to assess students' knowledge of the curriculum. Oh, one more thing, attach high-stakes for all those involved (e.g., administrators, teachers, and students) to the results of these tests to make certain this curriculum is taught and learned.

What isn't so simple is the impact that educational accountability has had on all those involved. There is still considerable a\mbiguity about the impact state testing has on instructional practices (Firestone et al., 2002). This study provides an insight and clarification on instructional practices used and factors influencing their use by social studies teachers who teach the same content that is tested on their state's high school graduation examination, but in some ways it leaves a number of questions open to further exploration.

References

Airasian, P., & Walsh, M. (1997). Constructivist cautions. Phi Delta Kappan, 78(6), 444-449.

Barksdale-Ladd, M. A., & Thomas, K. F. (2000). What's at stake in high-stakes testing: Teachers and parents speak out. Journal of Teacher Education, 57(3), 384-397.

Cimbricz, S. (2002). State-mandated testing and teachers' beliefs and practice. Educational Policy Analysis Archives, 10(2). Retrieved August 20, 2004, from http://epaa.asu/epaa/vl0n2.html

Eggen, P., & Kauchak, D. (2001). Educational Psychology: Windows on Classrooms. Upper Saddle River, N.J.: Merrill Prentice Hall.

Elementary and Secondary Education Act of 1965, Pub. L. 89-10.

Firestone, W. A., Monfils, L., Camilli, G., Schorr, R., Hicks, J., & Mayrowetz, D. (2002). The ambiguity of test preparation: A muiltmethod analysis in one state. Teachers College Record, 104(1), 1485-1523.

Good, T. L., & Brophy, J. E. (2000). Looking in Classrooms (8th ed.). New York: Longman.

Grant, S. G. (2001). An uncertain lever: Exploring the influence of state-level testing in New York station teaching social studies. Teachers College Record, 103(3), 398-426

Jones, B. D., & Johnson, A. F. (2002). The effects of high- stakes testing on instructional practices. Paper presented at the Annual Meeting of the American Educational Research Association. (New Orleans, LA, April 14-18, 2002).

Jones, M. G., Jones, B. D., & Hargrove, T. Y. (2003). The unintended consequences of high-stakes testing. New York: Rowman and Littlefield.

Marchette, F. (2003). Impacts of scheduling configurations on Mississippi biology subject area testing. (ERIC Document Reproduction Service No. ED482467)

McNeil, L. (2000). Contradictions of school reform: Educational costs of standardized testing. New York: Routledge.

Mississippi Department of Education. (2004a). Graduation Requirements. Retrieved August 20, 2004, from Mississippi Department of Education, Office of Student Assessment Web site: http:// www.mde.k12.ms.us/acad/osa/ne wgrad.html

Mississippi Department of Education. (2004b). Subject Area Testing Program. Retrieved August 20, 2004, from Mississippi Department of Education, Office of Student Assessment Web site: http://www.mde.k12.ms.us/acad/osa/sat p.html

Mississippi Department of Education. (2004c). Subject Area and FLE Graduation Requirements. Retrieved August 20, 2004, from Mississippi Department of Education, Office of Student Assessment Web site: http://www.mde.k12.ms.us/ acad/osa/grad.pdf

Natriello, G., & Pallas, A. M. (1999). The development and impact of high-stakes testing.(ERIC Document Reproduction Service No. ED443871)

No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425(2002)

Piaget, J. (1973). To Understand is to Invent. New York: Grossman.

Pressley, M., Rankin, J., & Yokoi, L. (1996). A survey of instructional practice of Primary teachers nominated as effective in promoting literacy. Elementary School Journal, 96(4), 363-384.

Skinner, R. A., & Staresina, L. N. (2004, January 8). State of the states. Education Week, 23(17), 97-153.

Vogler, K. E. (2002). The impact of high-stakes, state-mandated student performance assessment on teachers' instructional practices. Education, 723(1), 39-55.

Vygotsky, L. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, M.A.: Harvard University Press.

Wenglinsky, H. (2000). How Teaching Matters: Bringing the Classroom Back into Discussions of Teacher Quality. Princeton, N.J.: Educational Testing Service.

Yarbrough, T. L. (1999). Teacher perceptions on the North Carolina ABC program and the relationship to classroom practice. Ph. D. dissertation, University of North Carolina at Chapel Hill.

Zemelman, S., Daniels, H. & Hyde, A. (1998). Best practices: New standards for teaching and learning in American's public schools. (2nd ed.). Portsmith: Heinemann

Kenneth E. Vogler, The University of Tennessee at Martin

Appendix A

Appendix B

Appendix C

Appendix C

Appendix C

Appendix C

Appendix C

Appendix C

Copyright University of Northern Iowa Fall 2005

Source: Journal of Social Studies Research

More News in this Category