Assessment Forum

Creating and Administering a Primary Source Analysis

John Buchkoski, Mikal B. Eckstrom, Holly Kizewski, and Courtney Pixler | Jan 1, 2015

Hundreds of students pass through the introductory history courses of the University of Nebraska–Lincoln each semester with various backgrounds and skill levels. Although some of them have faced a primary source assessment, perhaps in the Advanced Placement Document Based Question Exam, our department had no universal assessment tool to evaluate student learning and skill development. In January 2014, William G. Thomas, our chair, and Margaret Jacobs, the chancellor’s professor of history, asked us to write and administer an exam for all introductory American history courses. Our goal was to construct new pedagogical tools that integrated more primary source analysis into our survey courses.

Collaboration was key to our success, and our different backgrounds and teaching experiences were vitally important in identifying the most significant components of history writing. We agreed that these components were a clear thesis statement, the number of sources used, organization, and analysis. We worked together to ensure that primary sources were similar for the pre- and post-1877 US exams in that they presented issues dealing with race, class, and gender. For example, we incorporated the Utmost Good Faith Clause from the Northwest Ordinance on the pre-1877 test and the Burton-Wheeler Act on the post-1877 test to demonstrate legal decisions regarding Native Americans and land. We chose sources that students were unlikely to have seen before and worked with the instructors to make sure the students would know the context in which the source had been created.

It is impossible to design a “one size fits all” exam that caters to the learning style of each student, but our mix of textual and visual sources helped make our exam more accessible. In order to prepare the students for the exam, the team led a workshop in each of the eight classrooms that focused on how to use primary sources. By working through a sample question and documents in the classroom, we modeled the best strategies for succeeding on the exam. We focused on how to write an effective essay, with a strong thesis statement and a cohesive structure guided by topic sentences. To ensure that online or absent students could review the information covered, the project team used Camtasia, a digital audio-recording tool, to record a podcast of the workshop, which we provided online along with a PowerPoint presentation.

The primary source assessment (PSA) team realized that the students’ teachers used a wide variety of teaching styles and presented very different course content. Each team member served as a liaison between two US history instructors and the PSA team. Faculty reaction to the PSA was varied; some instructors were enthusiastic, while others were initially skeptical. Two factors—clearly discussing the goals of the project and being available for further discussions—helped allay many of the faculty’s concerns. The instructors often found ways to adapt our lecture to fit the themes of the class and their teaching styles. Feedback from the instructors was particularly essential to the success of our workshops, as we were able to incorporate their suggestions over the course of eight lectures. This flexibility on our part helped us to adapt to the challenges that come with stepping into someone else’s classroom.

Because the members of the PSA team had varied historical interests and specializations, each contributed a different perspective to the exam. After choosing the American Revolution and the New Deal as the exam topics for each half of the survey, we eventually selected seven or eight sources. We decided to keep our questions somewhat simple in order to encourage argumentative thesis writing and broad use of evidence. For the first half of the survey, we asked, “Did all Americans benefit equally from the American Revolution?” and used sources covering women, Native Americans, and African Americans to provide a wide range of evidence. Likewise, for the second half of the survey, we asked, “Did all Americans benefit equally from the New Deal of the 1930s?”; we provided sources on African Americans, Native Americans, Mexican Americans, and working-class individuals. In order to remain objective in grading, we designed a rubric. We graded some of the exams together and frequently communicated about the grading process to ensure consistency.

The PSA team assessed the exams using four categories—thesis, sources, analysis, and organization—and tracked the results on a rubric (see sidebar). We then extrapolated various data sets from over four hundred exams. The HIST 110 (American history to 1877) classes averaged 83% for every category except the analysis category (77%). HIST 111 courses averaged higher (82%) in three categories (thesis, sources, and organization), but the analysis was still lower (79%). The averages for both American survey courses were as follows: thesis (83.4%), sources (83%), analysis (79%), and organization (82.2%).

We were pleased to discover that students taking the courses online scored comparably to those in traditional classrooms, leading us to believe that the PSA is effective in both environments. We were also pleased that the students focused on creating precise, argumentative theses and that almost every student provided a thesis statement. Additionally, most used at least six or seven of the documents we provided as evidence in support of their arguments. In the category of organization, a majority of students used a five-paragraph structure with topic sentences, which showed ability to effectively group evidence within a fluent argument. The PSA also indicated to our department the areas in which students could most improve. The analysis portion resulted in the lowest scores. We found that students referred to the sources in their essays but struggled to connect the context of the documents to their argument.

On the question about the American Revolution, in particular, we noted that students’ arguments often did not match the sources. Some students were reluctant to state that the Revolution did not result in total equality, incorrectly arguing that slaves and women became full members of the new Republic. Although students were more willing to be critical of the New Deal policies, we nevertheless encountered essays that argued for the fairness of repatriation and Indian removal. Many students relied on a predetermined narrative that was some variation of American exceptionalism and that clouded their ability to judiciously examine the sources we provided.

The exam also unexpectedly revealed that some students lacked a historical context for understanding race in American history. Documents and questions provided to students highlighted the entanglement of race with US history. During the workshops, the team discussed the interplay of race, class, and gender as a primary analytical tool for assessing the sources. Still, some students wrote that when African Americans are unable to receive Social Security, it is due to laziness, rather than to the historical legacy of the Jim Crow South and the institutional discrimination of the New Deal. When describing Native Americans in the New Deal Era, students adopted the “lazy Indian” trope. Others argued that the government should have taken Indian lands to boost the struggling economy. A few students also grafted current racialized debates onto the documents. For example, when analyzing documents on 1930s Mexican repatriados, one student wrote, “I think the New Deal was a little too fair for Mexicans wishing to return back [sic] to Mexico and take all their goods back. I don’t think illegal immigrants deserve this.” By couching their arguments in current issues of citizenship and migration, this student disregarded historical context. Finally, some students used problematic language found within the documents, such as the word negro.

As troubling as this was, the team agreed that the problematic essays were clearly not malicious. Rather, they seemed to be based on current political issues and a reluctance to criticize celebrated American policies. Although these problems were few in number, the department developed strategies to help students think about race as a construction, in and out of the classroom. One faculty member had advised against using the term negro in his syllabus and in class, and students from his class consistently used culturally appropriate language for African Americans in their essays. In spring 2014, the department discussed teaching about race in a previously planned workshop for faculty and graduate students. We were able to use our findings at this workshop to demonstrate the continuing need for critical race analysis in our classrooms.

Our department-wide efforts to bring a primary source assessment to our introductory classes produced mostly positive results. We were excited to discover that our survey-level students effectively produced structured, argumentative responses to primary sources. We also learned that, going forward, our focus should be on promoting critical analysis. The PSA team learned how to navigate the complexities of large-scale assessments and set the groundwork within our department for similar examinations in the future. Based on our results, the department is considering extending this assessment to other introductory surveys in the future.

The authors are graduate students in history at the University of Nebraska–Lincoln.

Grading the Primary Source Assessment

Thesis: 25%

22.5–25 A:

Makes a strong, precise argument.

20–22 B:

Makes a strong argument but speaks broadly.

17.5–19.5 C:

Makes a weak argument; recapitulates the question.

15–17 D:

Has an introduction but has no argument.

Under 15 F:

No thesis statement.

Number of Sources: 25%

22.5–25 A:

All sources

20–22 B:

Six–seven sources

17.5–19.5 C:

Four–five sources

15–17 D:

Two–three sources

Under 15 F:

Zero–one sources

Analysis of Evidence: 25%

22.5–25 A:

Analyzes sources in a sophisticated manner

20–22 B:

Analyzes sources in a somewhat sophisticated manner.

17.5–19.5 C:

Analyzes sources in a general manner.

15–17 D:

Analyzes sources poorly.

Under 15 F:

Lacks analysis.

Organization: 25%

22.5–25 A:

Clearly structured essay guided by thesis and topic sentences.

20–22 B:

Somewhat clearly structured essay guided by thesis and topic sentences.

17.5–19.5 C:

Essay is somewhat structured but lacks effective topic sentences.

15–17 D:

Essay is poorly structured.

Under 15 F:

Essay lacks cohesive structure.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Attribution must provide author name, article title, Perspectives on History, date of publication, and a link to this page. This license applies only to the article, not to text or images used here by permission.

The American Historical Association welcomes comments in the discussion area below, at AHA Communities, and in letters to the editor. Please read our commenting and letters policy before submitting.


Tags: Resources for Faculty Resources for Graduate Students Resources for History Departments Resources for Undergraduates K12 Certification & Curricula Teaching Resources and Strategies


Comment

Please read our commenting and letters policy before submitting.