Generative AI does not spell the end of the take-home essay
There is considerable concern about generative AI in higher education, particularly regarding writing assignments such as take-home essays. PPLE lecturer Pieter Lagerwaard argues in favour of allowing generative AI for take-home essays, but with different learning objectives.
There is widespread anxiety among university lecturers about students using generative AI (GenAI) when writing essays. There are even lecturers who want to throw GenAI off a cliff because it allegedly undermines critical thinking. Some are even asking aloud whether the university still has a right to exist.
Challenge
That fear is understandable. In addition to ethical issues surrounding climate impact, copyright, and racist bias, GenAI poses a huge challenge for writing assignments. Whereas plagiarism involves copying an existing text, GenAI generates a unique response every time. Research therefore shows that it is not possible to determine the use of GenAI with certainty.
I see around me various strategies being applied: 1) GenAI is (for the time being) ignored and its use is downplayed; 2) the essay as an assessment method is abolished; or 3) the essay is no longer completed at home, but on site – without access to the internet. I propose that another approach is possible: a home essay in which the use of GenAI is permitted, but with different learning objectives.
Experiment
For an essay assignment in a major first-year course at the University of Amsterdam (UvA), which I coordinate, we have, as an experiment, permitted the use of GenAI. Practical experience has shown that, above all, the learning objectives need to be revised: what skills do we want to and can we teach students in the GenAI era? And how can we assess these skills? The essay assignment is part of the Rhetoric course, within the PPLE bachelor’s programme. Originally, students were required to choose an inspirational speech and analyse it, but it soon became apparent after the launch of ChatGPT that it excels at this.
As the assignment accounts for fifty per cent of the mark, it was irresponsible to retain it in its current form. Certain writing skills – such as structure, grammar and spelling – are still important for a good essay, but with the use of GenAI, they have become a less important learning objective for this type of assessment.
Instead of an analysis, we asked students to formulate a rhetorical argument on a controversial topic. They could choose from two topics: the UvA’s collaboration with third parties, such as Shell, Huawei, or Israeli universities; or the UvA’s handling of the tension between academic freedom and social safety. Two new criteria were added, which GenAI struggles with: direct quotations from peer-reviewed articles, and the use of practical sources, such as policy documents and articles in Folia. GenAI can assist with both criteria, but it can be highly misleading in doing so, as it turned out.
Identical
There was a prior concern that the use of GenAI would result in identical essays. Certain arguments did indeed recur more frequently – social safety and academic freedom are not contradictory – but the essays proved diverse enough in quality and content to be assessable. The distribution of marks showed a clear so-called Bell curveThis is a graphical representation of data in which most values cluster around the mean, with fewer values occurring at the extremes (high or low). It forms a symmetrical, bell-shaped curve, with the peak at the mean..
Two findings from this experiment stood out. Firstly: despite warnings from us, many students overestimate GenAI’s capabilities during source research. It turned out that so many sources did not exist that, in consultation with the programme director, we had a spot check carried out. The problem proved to be substantial – sometimes involving several students per seminar group – but not to the extent that the assignment needed to be revised.
Students were found to make various errors: they referred to an existing article, but the sentence did not exist; they quoted an existing sentence, but referred to a non-existent article; or they quoted non-existent sentences from non-existent articles. These errors occurred at various stages of the writing process, including, for example, during the final grammar check.
These errors are unique to GenAI – so-called hallucinations – and scientists and journalists also struggle with them. For instance, the new rector of Ghent University referred in her inaugural speech to a quote by Einstein that he never made. The former editor-in-chief of NRC used citations in 15 of his 53 blog posts that did not exist.
References
It seems to me that the skills with which scientists and journalists struggle should, in fact, be taught at university – and the take-home essay appears to be a suitable assessment method for this. To assess this learning objective, there should not be a minimum number of references, as is often the case now, but a maximum number, with the lecturer checking every single source.
The second finding: whilst GenAI is good at formulating general pros and cons and taking a political stance, it proves less helpful when it comes to a specific argument linked to practice – in this case, the University of Amsterdam (UvA).
Just as with academic sources, GenAI starts to ‘hallucinate’ when it comes close to practical applications: students quoted statements from UvA staff members who turned out not to exist; made incorrect references and paraphrases; or referred to general policy, whilst the reference actually pointed to an individual profile page.
Self-reflections
The students’ self-reflections therefore revealed that the use of GenAI could disrupt the writing process, take up extra time and cloud or distort the argument. Many students indicated that they ultimately chose to use GenAI solely as a source finder, sparring partner or editor – and intended to continue doing so in the future.
Good essays also often remained closely tied to UvA debates and took a stance on them. Students who criticised or embraced UvA guidelines, or cited UvA controversies, were more precise and convincing than the often generic GenAI prose. A connection to real-world practice is therefore a useful yardstick for determining whether a student has immersed themselves in the subject matter and has used GenAI wisely.
Review the learning objectives
An on-site essay and a take-home essay are complementary, in my view. It is important that students learn to write, but without access to the internet, an on-site essay offers no opportunity to learn ‘in the wild’ how to find, assess, and use sources – and in this way develops the much-needed AI literacy.
A link to real-world practice is perhaps more common in the social sciences than, for example, the humanities, but historical and philosophical essays can also be linked to practice and make use of historical, archaeological, or legal sources. Ethical concerns regarding authenticity are legitimate, but that does not mean that the at-home essay no longer tests important skills – perhaps now more than ever.
Pieter Lagerwaard is a lecturer in the Politics, Psychology, Law and Economics (PPLE) programme at the UvA.
