Initiative for safe Artificial Intelligence at UvA marks 1 year anniversary: “We need to get started nów”

Growing concerns
By ‘science fiction-like’, Leonard is referring to future scenarios in which autonomous AI systems, that make decisions independently without human intervention, surpass human intelligence, called ‘AGI’ among scientists. Think of fully autonomous driving cars. Or language models that could, as Leonard predicts, “automate most of my current research activities in the future”.
 
With the rapid breakthroughs of these AI language models like ChatGPT over the past two years, UvA scientists suddenly became aware of how fast developments within artificial intelligence can go, Leonard says. At the UvA, concerns about AI plagiarism by students grew last year, and UvA researchers signed a letter along with AI giants like Elon Musk warning of a scenario in which AI could cheat humanity’s survival.
 
It provided Leonard and several other PhD students at the UvA with a “wake-up call” about the danger of this feverish progression, including the scenario described earlier in that letter. “The possibility of future advanced AI systems like AGI causing devastating and irreversible damage is real. And that these systems will be here by 2027 is not unlikely. This presents many opportunities, but we must also ensure that our understanding of these systems and our ability to control them keeps pace with these developments.”

Lots of competition
For these PhD students, this was the go-ahead to bring students and researchers together to seriously discuss AI and its security issues. A bold move on the Science Park campus where competition within AI research is high, according to Leonard, and which can sometimes hinder collaborative efforts.
 
“At the UvA and many other institutions, there is a strong emphasis on technical breakthroughs that enhance the capabilities of AI,” Leonard notes. “While that is important, the security challenges of AI require large-scale, interdisciplinary collaboration and perhaps even a redefinition of what we mean by progress.”
 
Yet AISA got that collaboration between students and researchers off the ground within a short time. Members of the initiative organised several lectures and panel discussions on brand-new, cutting-edge security research. A big step, since open academic debate on ‘science fiction-like’ AI systems was unheard of at the UvA until a year ago, according to Leonard.
 
How did AISA get this done so quickly? According to Leonard, by building a community that approaches the topic from different angles. “AI security is complex and requires both technical, ethical and societal considerations. There are numerous scenarios to explore, from malicious to unintentional misuse of AI systems. Our goal is to bring together a diverse community that addresses these challenges.”

Young field
The challenge was that AI and safety as a discipline has only been taken seriously for the past two years and therefore did not yet feature in UvA’s AI courses. How do you attract enthusiastic students if they have never heard of the subject before? Leonard decided to approach lecturers personally, asking if AI Safety could have a place in their courses.
 
Curriculum development at the UvA usually takes place behind the scenes between lecturers and committees. So Leonard wasn’t sure if a similar initiative had succeeded before, but AISA succeeded. “I was pleasantly surprised by how open lecturers were to our input. It is such a young field, so the fact that AI safety is already included in the curriculum is a great achievement,” Leonard said.
 
The focus on AI and safety in the curriculum, as well as word of mouth on Science Park, paid off. AISA grew from a handful of researchers to a group of 60 students and PhD students. Something Leonard hadn’t expected given how young the field still is and that still few UvA researchers are formally engaged in it: “I did expect researchers to be involved, but was surprised at how many specifically students were interested in the topic. The biggest concerns about AI security really came from them.”
 
Meanwhile, AISA is also grooming the next generation of researchers. AI students can turn to the initiative for research support. This is much needed, as research on AI and safety often raises complex ethical issues, Leonard says. “To understand AI risks, we need to work with sophisticated models that can mislead us,” he explains. As an example, he cites a language model that can convincingly sell lies to a user. Similar research, he says, raises ethical dilemmas similar to those of virological research where viruses are made more dangerous. “Increasing the capacity of AI models also increases the risk of them causing unintended harm. That is sometimes a tricky trade-off,” he admits.
 
AISA hopes to launch a mentoring programme in the coming college years to pair experienced AI researchers with UvA students who want to study AI and security. He hopes this will help more students with their research and stresses that given the speed at which super-intelligent AI is upon us, educating the next generation of researchers is crucial. “I worry whether we are going to be in time to manage the risks of autonomous, super-intelligent AI. Society is not yet prepared for that.”
 
Yet Leonard remains optimistic. He believes we can avoid a doomsday scenario, provided we take action. “During the COVID pandemic, we were able to move quickly once the urgency was clear. We can do the same with AI security threats, but we need to act now.”