Painting by Guillaume Lethière titled Oath of the Ancestors shows two generals involved in the Haitian war of independence, Alexandre Pétion and Jean-Jacques Dessalines

AI and the Haitian Revolution

To better understand generative AI, I created a new assignment in my HIST 407 course this past semester. The course is on Slave Rebellions and Resistance in the Atlantic World. At the end of a unit on the Haitian Revolution, I asked my students to write an essay on one of the main themes in this history and then produce an AI-created essay on the same topic. Students would then reflect on how their essay compared with the AI essay.

I thought this assignment might lead to a discussion on the similarities between human- and AI-generated content. But that’s not what happened. Instead, I found that AI consistently produces fake sources. And this has shaken my trust in this new technology.

First, let me explain the writing assignment. It was a summative exercise meant to assess students’ content learning and their analytical skills in evaluating primary sources. I also hoped that it would help students to critically engage with AI. Here’s the assignment:

The Haitian Revolution and AI Analysis

This is a three-part assignment. Submit all three components in class.

Write a 1,000 word essay that analyzes one of the themes below on the Haitian Revolution. To do this, make an argument about this theme using specific evidence from the readings. Be sure to quote from relevant primary sources that are found in David Geggus’s book, The Haitian Revolution: A Documentary History. And make references to Laurent Dubois’s book, Avengers of the New World: The Story of the Haitian Revolution.

  • Women in rebellion
  • Freedom after the Emancipation Proclamation
  • Foreign invasions and the international context
  • The War of Independence
  • Toussaint Louverture’s effectiveness as a leader
  • Origins of the slave insurrection
  • Organization and communication in the rebellion

Select a generative AI (such as ChatGPT, CoPilot, Claude, etc.) and input the above prompt. Print this AI response. Then, write a 250-500 word assessment of the two essays (yours and the AI), taking care to consider what was included or excluded in both. Based on your knowledge of the Haitian Revolution, how accurate or persuasive is the AI essay?

Write your essay before generating the AI essay. In total, your submission will have three components: your analysis of one of the themes from the Haitian Revolution, the AI essay, and your reflection on AI and historical analysis. Please identify each of the three components clearly.

When I began to assess the student essays, I neglected to read the AI essays. I didn’t have time for them; after all, my job was to evaluate the twenty-four student essays. But a few papers into the class set, I began to look at how AI used sources. The AI essays included parenthetical citations and sometimes direct quotes. But something seemed wrong about them. I began to look up every citation, and I found that they were all incorrect, every single one of them. Most often, the page number cited had nothing to do with the content in the essay. Sometimes the AI would even cite a page number that was larger than the number of pages in the book.

Even more troubling was that the quotes were nearly always fabricated. I did find one primary source quotation that was accurate. But all the others were fraudulent. These quotes, of course, seemed accurate, which is why most students didn’t bother to check the citations in the AI essays. By the way, students used a variety of AI apps, but mostly ChatGPT.

The AI also had a hard time distinguishing primary sources. The Geggus book is a collection of documents on the Haitian Revolution. The AI, however, frequently treated Geggus as a scholarly source, attributing arguments and quotes to this author in the same way that it referenced Laurent Dubois’s monograph on the Haitian Revolution. To be fair, Geggus includes an interpretative essay at the beginning of his book and has editorial introductions to the documents that are interpretive. But the AI didn’t seem to know how to use any of the particular documents in the book. From the AI’s perspective, all of the content in the Geggus book was of equal value. But from a historian’s perspective, we value primary sources that are closest to the events described. I wanted students to use the primary sources in Geggus’s book as evidence for their claims, which they did. The AI essays could not do this.

In addition to producing fraudulently sourced essays, I noticed several historical errors in nearly every essay. Some of the errors reflect the way that AI “scrapes” the internet for data. Most essays asserted that Haiti was the first Black republic, which is a common assertion, but not accurate because it was not a republic at the outset. And neither of the two books made this claim. Something else I noticed in the AI essays was their tendency to use “five-dollar” words, such as “acumen.”

All in all, students seemed to appreciate this assignment. We have been talking about AI and trying to make sense of it. Several students confessed that they had never used it before. But only a couple of students bothered to check the AI citations. So, I emphasized this point when I passed back their papers. The AI responses seem authoritative and credible, but they ought to be carefully scrutinized.

I will probably do this assignment again. I came away from it with a better understanding of how AI works, and I think this will help students to be more critical of AI going forward.

Lastly, this assignment raises troubling questions about how AI is used. I can imagine that there are tasks for which AI is well-suited. But it is alarming to me that generating fake sources seems to be a feature of AI, not a bug. We teach students that they need credible evidence to support their claims and that they must cite their sources to gain a reader’s trust. AI failed to do both accurately.

I also got the sense that AI produced responses that were intended to satisfy the human user. Take, for instance, the prompt on Women in Rebellion. The AI fabricated evidence of women as soldiers fighting in the Haitian Revolution because it assumed that rebellious acts must be violent and expressed in an insurrectionary way, much like Nat Turner’s rebellion. The AI did not say that it could not answer the question sufficiently. Nor did it consider other ways to conceptualize rebellion or other ways that rebels might have acted. My students could do this, of course. And my students didn’t fabricate sources.

As a result of this exercise, I am even more alarmed about the growing belief that AI can produce trustworthy content. Although this was one writing assignment, these so-called “hallucinations” are becoming more prevalent.

Image Credit: Guillaume Guillon-Lethière, “Le serment des ancêtres” (“Oath of the ancestors”) (1823), Public domain, via Wikimedia Commons