UvA investigates impact of AI watermarks on its own chatbot
The UvA is investigating the impact of the introduction of watermarks for AI-generated text on its own chatbot, UvA AI Chat. AI providers are increasingly adding such invisible markers to texts generated by their language models in order to comply with new European legislation on artificial intelligence.
In the rapidly evolving digital landscape of generative AI – a dynamic reality full of unprecedented possibilities – the university is navigating cautiously at the intersection of innovation and integrity.
Sometimes you can tell immediately that a sentence was written by AI. Like the one here above.
But that is by no means always the case. As chatbots become increasingly convincing at writing like humans, demand is growing for watermarks: ways of identifying AI-generated content as such. OpenAI recently introduced the textGrain watermark, which will be invisibly embedded in ChatGPT’s text. Competitor Anthropic previously introduced a similar system for Claude. These measures are also intended to comply with new EU transparency rules governing the use of artificial intelligence.
Identifying AI-generated text using a watermark is – like using other AI detectors – not a foolproof way of determining whether artificial intelligence has been used.
Short AI-generated texts, in particular, often go undetected. Moreover, the detection rate drops quickly when users edit the generated text. With textGrain, changing just 10 per cent of the words in a text was enough to reduce the likelihood of detection from 92 per cent to 66 per cent.
UvA AI Chat
These developments may also have implications for UvA AI Chat. After all, the university’s internal AI tool runs on external language models such as ChatGPT and Claude. If these tech companies add watermarks to their texts, they will theoretically be carried over automatically to the UvA system.
Whether those watermarks arrive intact, however, remains to be seen. This has to do with the way AI tools generate and watermark text. When formulating sentences, language models repeatedly predict the most likely next word from a list of options, much like an enormous decision tree. An AI watermarking system influences these word choices using a secret key. A human reader will notice nothing unusual about the text, but beneath the surface, a mathematical pattern emerges that can later be identified by a special detector, simply by examining the word choices made by the model. According to a university spokesperson, the UvA is currently investigating whether watermarks added by AI providers also remain intact “when processed by the UvA AI Chat software”.
Legal question
The university is also examining a legal question: under the law, does the UvA qualify as an “AI provider” because it operates its own chat environment, or is it merely an “AI user”? The European AI Act is not yet entirely clear on this point for organisations that use models developed by others. If UvA AI Chat turns out to qualify as a “provider”, the university may need to take additional measures to comply with EU legislation.
The European AI Act also requires users to know when they are communicating directly with AI. In this respect, the UvA’s AI tool already complies with the guidelines.
