AI detectors flag theses as heavily AI-generated: how problematic is that?
Almost 70 per cent of the text in the literature review chapters of economics bachelor’s theses is flagged as AI-generated by the AI detector Pangram. This is based on a sample analysed by Folia. But how problematic is it that the AI detector is flagging so much of the text as AI-generated?
“Of course, I’d be happy to help you with your literature review!” For students struggling with their thesis, help is never far away. With tools such as ChatGPT, Claude and UvA AI Chat, artificial intelligence is increasingly becoming a study partner. Chatbots can help with an outline, write paragraphs and chapters, or edit texts. Yet it often remains unclear how frequently and when students use AI when writing their theses.
To gain more insight into this, Folia sampled the literature review and background chapters of 50 bachelor’s theses from the Faculty of Economics and Business (FEB), using AI detection programme Pangram (see methodology). Unlike theses from many other faculties, such as the Faculty of Humanities (FGw), bachelor’s theses from this faculty are publicly available through the university’s thesis repository. Research suggests that Pangram is capable of detecting traces of AI in large amounts of data, although the programme can be inaccurate at the individual level. An individual score therefore does not provide conclusive evidence that a student has used AI.
In total, Folia analysed 50 bachelor’s theses from 2026 from the Faculty of Economics and Business (FEB). The theses were taken from the UvA’s public thesis repository.
Using the AI detection programme Pangram, Folia then examined what proportion of the text was flagged by the software as AI-generated. Pangram itself claims that it incorrectly flags only 1 in 10,000 texts as AI-generated. Independent studies are more critical, but put the false-positive rate at no more than a few per cent.
For each thesis, Folia analysed the chapters containing the literature review. This was chosen because of Pangram’s character limits and because almost every bachelor’s thesis includes a literature review.
For comparison, ten bachelor’s theses from 2021 were also analysed, from before generative AI tools such as ChatGPT became widely available. In none of these theses did Pangram flag any text as AI-generated.
In 48 of the 50 theses, Pangram flagged at least part of the analysed text as AI-generated. On average, Pangram flagged 69 per cent of the analysed text as having been generated by artificial intelligence. One in four texts was classified entirely as generated content, with an AI score of 100 per cent. The results show, at the very least, that detection software flags a significant proportion of the text in many theses as AI-generated. But is that a problem?
Permitted
First, to be clear: UvA students are now permitted, under certain conditions, to use artificial intelligence when writing their theses. The rules vary by faculty and even by degree programme. The Faculty of Humanities (FGw), for example, only allows the use of AI as a sounding board or translation aid; generating continuous text is in principle considered fraud. FEB, by contrast, gives students broader permission to use AI during the writing process, “provided that throughout the thesis process it is ensured that the student has mastered the material at the required level.”
Marc Salomon, dean of the Amsterdam Business School within the Faculty of Economics and Business, is therefore not surprised that the AI detector flags so many theses. “We teach students throughout the curriculum how to use AI responsibly. That naturally also includes conducting research with AI in a responsible way.”
Transparency
According to the university’s guidelines, students must nevertheless “be transparent about how they have used generative AI”. That does not always appear to happen. Some students explicitly and extensively describe their use of ChatGPT, for example, in an “AI statement” in their thesis. In other theses that Pangram flags as largely or even entirely AI-generated, however, there is an originality statement in which the student explicitly declares that they have not used generative AI.
This does not immediately concern Salomon either: “A detector score is not hard evidence of improper use. Such a detection programme can also flag text that students have written themselves but have only run through a spelling checker such as Grammarly. Many students do not consider spelling and grammar checkers to be ‘AI use’.”
Desirable?
A high AI score therefore does not necessarily mean that a student has done anything wrong. But that still leaves the question of whether it is desirable for an AI text generator to take care of a large part of a thesis – after all, the most important assignment of a degree programme.
According to Rens Bod, professor of computational and digital humanities, this depends heavily on the field: “Look at the experimental sciences, such as biology or physics. If a student has spent weeks in the lab, carried out all the experiments themselves and meticulously documented all the data, is it really so terrible if AI lends a hand with writing it up? In the humanities, by contrast, formulating the text and carefully constructing an argument are at the heart of the discipline. There, things are different.”
He also sees benefits to using artificial intelligence in the writing process: “For example, there are many people in the sciences who simply cannot write well. For them, generative AI can be a solution that helps them present their findings in a more accessible way. In that sense, it is almost a form of democratisation of writing.”
After university
Moreover, the university is preparing students for a reality in which AI is becoming increasingly difficult to imagine without. In professional practice, people often do not care whether a policy memo, email or quarterly report was partly produced with the help of ChatGPT. What matters most is the result. That is precisely why Bod believes it is important to teach students during their studies how to use AI responsibly: “It is also important for students to learn how to use AI responsibly. At the same time, I do think it is useful for students to use their thesis as an opportunity to do something as much as possible entirely on their own at least once.”
According to Salomon, the Faculty of Economics and Business therefore takes several measures to check whether students can account for their own work and choices. During the thesis process, FEB students have four progress meetings in which their supervisor assesses the student’s own knowledge and intellectual contribution. In addition, students must ultimately defend their thesis orally.
Faking will become difficult
Bod believes that measures like these can certainly discourage students from outsourcing large parts of their work to artificial intelligence: “At some point, you reach a stage where you have to put so much effort into faking your work with generative AI that it is no longer worth it.”
He also believes that students who do so will quickly be caught out during oral assessments: “They say that writing something once is the equivalent of reading it ten times. So if you did not write a text yourself, it often does not stick in your head as well.” At the same time, he regrets that these checks are necessary: “I think it is a real shame that this forces lecturers to become such obvious police officers. That is quite awful.”
