In this lab, algorithms help unravel criminals’ hidden messages
To find out exactly what happened in a crime, experts at the Netherlands Forensic Institute (NFI) are increasingly using artificial intelligence. At the Science Park, the first lab has now opened, AI4Forensics, where the NFI is collaborating with the UvA. “With statistics, we can figure out what images contain hidden messages.”
Artificial intelligence is on the rise, including in the criminal world. Using algorithms, criminals are increasingly sending hidden messages in images, texts, audio files, and videos. As a criminal, you don’t have to be smart to send messages with steganography, or in other words, by sending information through innocent objects. Via Google, steganography software is easy to find.
The Netherlands Forensic Institute (NFI) is also using artificial intelligence to support experts in forensic investigations. That is why the NFI opened the AI4Forensics at the Innovation Center for Artificial Intelligence (ICAI) in Lab42 in mid-March. The new lab is led by Professor of Multimedia Analytics Marcel Worring and Associate Professor of Forensic Data Science Zeno Geradts, where four PhD students and one postdoc are also working on artificial intelligence in forensic investigations.
Invisible ink pen
PhD student Meike Kombrink is one of them. She works on detecting hidden digital messages. “Steganography in the physical world can be compared to writing with an invisible ink pen,” says Kombrink. “The message cannot be seen with the naked eye and only becomes visible when you shine a light on it.”
You can hide a message in an image by adjusting the binary values of colors, changing only the very last zeros and ones such that they also encode letters. This changes the color intensity of a pixel a minuscule amount.
Another method is to summarize images, such as in a compressed image. “Like in a text, you can leave the core sentences untouched, but change the examples. The messages are hidden in those examples.”
The third way is to train AI to hide the message itself in an existing image, video, audio file or text. Or to create the image itself with the message in it.
Where does the NFI start looking? You can’t possibly look at everything. “One of the giveaways is communication beforehand and afterward,” says Kombrink. “Criminals need to know from each other that a message is coming and how it’s hidden, so they communicate about it.”
The second step is to reveal the message. To do that, Kombrink uses a self-created data set, among other things. She explains how she trains the algorithm using the dataset. “Using statistics, we can find out what images contain a hidden message.”
“Eh” and “so”
Artificial intelligence can also help compare voices in voice recordings. The algorithm cannot tell whether a voice belongs to a particular person. It only gives a measure of probability. This allows a language expert at the NFI to make a better estimate.
For the NFI, the existing automatic speaker comparison is not always optimal, says PhD student Eleni Sergidou. “Standard software is usually trained with millions of English-language YouTube videos,” Sergidou says. “It also uses only the sound of the voice and not other features. For NFI cases, the recordings are often Dutch and of poor quality, such as from tapped phone lines. Criminals may also use voice distortion devices if they know their lines are being tapped. For those cases, it is important to listen not only to sound but also to other characteristics of the voice.”
That’s why Sergidou also focuses on the words a person uses for forensics. Each individual not only has their own characteristic vocabulary but also uses words that have no meaning such as “eh” and “so” in a characteristic way. By including that information, the algorithm as a whole can make more distinctions. Sergidou is working on a model that uses both sound and word choice.
The model needs to be individualized to a specific case. Sergidou comments: “It’s not plug-and-play yet. For each case, we see how well the algorithm works. For example, if there are phone calls with a lot of noise, the algorithm performs differently than with studio recordings.”
Ultimately, the goal is to use Kombrink and Sergidou’s AI models in the forensic search engine Hansken, which is used in criminal cases. Then you can find one voice in thousands of recordings or check for hidden messages in all the images, texts, videos, and audio files from a case in one fell swoop.
But the technology is not that far yet. Kombrink is now working on making visible which messages are being exchanged. “I could send my friend a shopping list in stego that includes tampons if I don’t want everyone reading along,” she jokes. “But that doesn’t make it interesting information for criminal justice. The goal in four years is to not only track down the messages but to decipher them.”
