Recent Additions to the Zotero Library: Schneier, Whittaker, Underwood

This post is a part of a continuing series that highlights noteworthy additions to the Geneseo Center for Digital Learning’s public Zotero collection of resources on AI. Posts in this series are group under the category Zotero Additions.

Trust and Accountability

Two recent additions to the collection look at questions of trust and accountability. Under the forthright title, “Can you trust AI? Here’s why you shouldn’t,” Bruce Schneier, Adjunct Lecturer in Public Policy, Harvard Kennedy School, and Nathan Sanders, an affiliate at the Berkman Klein Center for Internet & Society at Harvard, point out that the major AI tools—those offered by companies such as Meta, Microsoft, OpenAI, Facebook, and Google—have been configured first and foremost to serve those companies’ interests. Sure, we’re used to the fact that these companies’ other products are ultimately self-serving, but what’s different about their AI products is that these are rapidly finding their way into our everyday routines in the form of interactive assistants designed to carry out a multitude of tasks for us. A genuinely useful assistant “will have to really know you. Better than your phone knows you. Better than Google search knows you. Better, perhaps, than your close friends, intimate partners and therapist know you.” With this knowledge, Big Tech will be better positioned than ever to surveil, deceive, manipulate, and misinform you.

Imagine asking your chatbot to plan your next vacation. Did it choose a particular airline or hotel chain or restaurant because it was the best for you or because its maker got a kickback from the businesses? As with paid results in Google search, newsfeed ads on Facebook and paid placements on Amazon queries, these paid influences are likely to get more surreptitious over time.

If you’re asking your chatbot for political information, are the results skewed by the politics of the corporation that owns the chatbot? Or the candidate who paid it the most money? Or even the views of the demographic of the people whose data was used in training the model? Is your AI agent secretly a double agent? Right now, there is no way to know.

There is no way to know because the algorithms and training data for these AI systems aren’t publicly shared. (Despite its name “OpenAI” is not open about either of these.) And because we’re often in the dark about how these companies use the personal data we hand over willingly as the price for the “free” services these companies offer.

Not surprisingly, then, Schneier and Sanders conclude that the best defense against large-scale surveillance, coercion, and political manipulation by these increasingly prevalent, increasingly inescapable tools is to hold their makers accountable through robust laws and regulations designed to ensure privacy and transparency.

Meredith Whittaker’s short paper “The Steep Cost of Capture” provides a helpful account of how we got to the situation Schneier and Sanders describe—how it came to be that “Modern AI is fundamentally dependent on corporate resources and business practices, and our increasing reliance on such AI cedes inordinate power over our lives and institutions to a handful of tech firms.” Whittaker, who wrote the paper during her time as a professor at New York University, is currently president of the non-profit Signal Foundation, which maintains the free, encrypted messaging app Signal. Her essay draws parallels between, on the one hand, the way large corporations currently shape not only AI products but AI research and the discourse around it, and the way the Cold War era military shaped the research and discourse around early computing. The “cost of capture” for researchers partnering with or dependent on the defense department in one case and Big Tech in the other is the inability to control the research agenda or to voice dissent.

AI and Liberal Education

Academics in the humanities disciplines, including those in digital humanities, have been among AI’s fiercest critics. An exception is Ted Underwood, who teaches in both the School of Information Sciences and the English Department at the University of Illinois, Urbana-Champaign, where he uses machine learning to analyze large bodies of textual data.

Since the launch of ChatGPT in 2022, Underwood has written a number of posts for his blog “The Stone and the Shell” trying to sort out the promise of AI from the kinds of concerns raised by Schneier, Sanders, Whittaker and others, as well as trying to reckon with its apparent inevitability. These are all worth reading, but here we’ll just highlight his 2023 post “Liberally educated students need to be more than consumers of AI.” In it, he argues that the conversation around AI in higher education is “confined right now by a discourse that treats models as paper-writing machines” and that’s focused on the challenge to faculty of distinguishing genuine student work from cheating. “But the challenge that really matters here is a challenge for students,” he maintains, “who will graduate into a world where white-collar work is being redefined.”

This is hardly an unusual point by now. But whereas the next step for many writers commenting on the need to give students AI skills is to turn to the importance of “prompt engineering,” Underwood goes in a different direction:

A decade from now, in short, we will probably be using AI not just to gather material and analyze it, but to communicate interactively with customers and colleagues. All the forms of critical thinking we currently teach will still have value in that world. It will still be necessary to ask questions about social context, about hidden assumptions, and about the uncertainty surrounding any estimate. But our students won’t be prepared to address those questions unless they also know enough about machine learning to reason about a model’s assumptions and uncertainty. At higher levels of responsibility, this will require more than being a clever prompter and savvy consumer. White-collar professionals are likely to be fine-tuning their own models; they will need to choose a base model, assess training strategies, and decide whether their models are over-confident or over-cautious.

That’s a prescription for a level of technical understanding required rarely, if ever, right now for liberally educated students as a whole. It will be interesting to see how many institutions are ready to follow it.

Photo credit: “Trust” by mmrtnt is licensed under CC BY-NC-ND 2.0 .