Han van der Maas | What if AI turns out to be really good at science?
Columnist Han van der Maas is one of the fools who use AI for genuinely intellectual tasks. For what are scientists supposed to do when systems like Gemini, Claude and ChatGPT can churn out entire models and papers in a matter of hours? “I haven’t submitted it yet, because I’m confused. What has happened here? Who is the author? What if everyone starts doing this?”
There are few AI abstainers left at the University of Amsterdam. With Gemini integrated into Google Search, taking the first step is easy enough. Most people use generative AI mainly for routine academic tasks: rewriting an email or abstract, coming up with titles, creating a figure. But there are also fools who deploy this technology for genuinely intellectual tasks. I am one of those fools, and this piece is about them.
Not written a single word
My latest transgression was asking Claude to set up a computer simulation to test an explanation for a particular empirical pattern in reading data. Claude was happy to do so, but also pointed out that a mathematical approach might be possible. “Go ahead,” I said, while playing a game of blitz chess on Lichess. We finished at roughly the same time, after which Claude told me the model could also be fitted directly if data were available. I had the data, and two games later an algorithm had been written and applied. “Shall I put this into a document for you?” “Go ahead, in LaTeX and APA format.” Another game later, the paper was ready. Great work, I concluded in the chat, but Claude disagreed. The fit was “at best mediocre,” the model relied on several questionable assumptions, and alternative models might perform better. “Oh, right,” I said, “put that in the discussion, and could you make the whole thing a bit shorter while you’re at it?” And then it was done, after about three hours. Ready to submit, with me in the acknowledgements or perhaps as second author. I hadn’t written a single word.
I haven’t submitted it yet, because I’m confused. What has happened here? Who is the author? What if everyone starts doing this? A certain Nicholas Polson published around 250 papers and more than 10 books in nine months. Nobody is particularly happy about that, but it is certainly bizarre.
Queen of the sciences
I am not the only one who is confused, because this is just one of many such anecdotes. All around me, colleagues are producing papers of this kind. Worse still, sometimes no academics are involved at all. In recent months, OpenAI first claimed to have solved one of the Millennium Prize Problems, and then went on to publish solutions to hundreds of other open mathematical problems. These are not ordinary problems: they require deep mathematical insight and enormous creativity. Mathematics, the queen of the sciences, has been wrested from mathematicians. This, too, is deeply confusing. If hundreds of outstanding problems were suddenly solved in other fields, cancer would be curable, the energy crisis would be over, and depression would be a thing of the past. Surely we can hope that such breakthroughs in mathematics will have some practical value.
The question of whether this amounts to superintelligence is becoming increasingly difficult to ignore. The term has now been embraced by a prominent American politician who repeatedly reminds us that super-stupidity is our greatest concern. One possible answer to the question draws on the distinction between fluid and crystallised intelligence. The fluid intelligence of AI systems still seems weaker than that of humans, although this human stronghold is crumbling rapidly. There can be little doubt, however, that we are witnessing “super-crystallised intelligence.” From chess to composing, from mathematical reasoning to writing, AI is delivering astonishing performances.
Lockdown
For me, this AI revolution feels like the first COVID lockdown in March 2020. Back then, too, we had to reinvent the university. How do we work alongside AI agents? How do we redesign the publication process and maintain control over science? And how do we rethink scientific education? The master’s thesis, the crowning achievement of a degree, has been knocked off its pedestal. What, exactly, should students learn? How may they – and how should they – use AI?
I know that all kinds of task forces and policy committees have been set up, and I am involved in some of them. But you cannot make new rules in the middle of a transition. Just as during the COVID crisis, we have to improvise. We are making it up as we go along. I also notice that we increasingly need different kinds of gatherings to discuss these developments: self-help groups for academic fools where we can share our bewilderment. As director of the IAS, I would be happy to make room for such meetings. On Fridays, we will be organising AIA gatherings: AI Anonymous.
