Artificial intelligence can spy art forgeries, UO study finds

Artificial intelligence can spot art forgeries, study finds

Training computers to spot fractals

The tool harnesses machine learning to recognize visual characteristics such as fractal patterns found in Pollock’s unique “poured” compositions. 

Machine learning is the ability of computer systems to learn patterns found in a training dataset and then make decisions without any additional programming. Fractals are naturally occurring patterns that repeat at increasingly small scales, visible in trees, clouds, snowflakes and coastlines.

Pollock’s unusual technique of pouring out paint while walking around his canvas created similar patterns within his art, making it an ideal candidate for computer analysis, Taylor said.

Julian Smith and Caleb Holt, both UO doctoral graduates, trained an artificial neural network to identify those patterns by feeding it images from Taylor’s personal collection of nearly 600 digitized artworks, the largest digital collection of Pollock paintings, imitations and other abstract works ever assembled, according to their study.

But it takes thousands of images to train a neural network effectively. To overcome that challenge, Holt and Smith devised a novel machine learning strategy for image study, breaking each picture into smaller “tiles” at a variety of different scales. 

Because of their fractal nature, Pollock’s paintings are recognizable whether looking at the full artwork or just a small corner, Holt said. That made it possible for the machine learning program to identify the same patterns in each section of a painting as in the whole.

“We could take an image of a Jackson Pollock painting and chop it up,” he said. “Now we could have 20,000 images, as many as we need to train a machine learning model reliably.”

The resulting tool, while highly accurate, functions as a “black box,” offering no visibility into how it arrives at its conclusions. The team conducted rigorous testing to determine not only whether it works but why it works, Smith said.

“I can see that it’s providing a result, but I don’t really know what’s going into all of the little decisions being made,” he said. “The worry is that your AI is doing its job but for the wrong reasons. We found that it’s doing exactly what we thought it should.” 

“We could take an image of a Jackson Pollock painting and chop it up,” he said. “Now we could have 20,000 images, as many as we need to train a machine learning model reliably.”

Caleb Holt, UO doctoral graduate

Because the tool can receive data and deliver results without human interference in its decision-making process, it has the potential to help end the Pollock-related fraud scandals that “come along routinely like trains” in the art world, Taylor said.

“A lot of the controversy around art authentication is whether the humans involved are biased or have a motive,” he said. “We wanted to be able to feed things in with no human contamination.”

But computers can’t do it alone

The researchers emphasize that their authentication technique is not intended to be used in isolation and won’t be putting any art experts out of a job. Rather, when coupled with other techniques such as materials analysis and visual inspection, AI can provide a scientific complement to human expertise.

“Some of the greatest Pollock scholars sometimes get it wrong; the human eye gets it wrong,” Taylor said. “If AI can help sort these cases out pretty quickly, we will be doing the whole art world a service. It’s a really beautiful opportunity for an art-science collaboration.”

Taylor has been using computers to analyze Pollock’s art since the 1990s, when he realized the paintings contained the same fractal patterns he had been studying in nature, a style he dubbed “fractal expressionism.”

“Pollock managed to distill the essence of nature and put it onto a canvas,” he said. “I believe he would be absolutely amazed that a computer now studies his artworks and can separate his master works from imitations with remarkable accuracy.” 

But the AI tool isn’t just for settling authenticity disputes. The researchers say their study provides scientific evidence that Pollock’s artistic signature is quantifiably different than that of other artists who mimic his poured paint technique, confirming that Pollock’s contributions to modern art go beyond merely popularizing the pouring technique.

It also raises thought-provoking questions about whether artificial intelligence can appreciate art, Taylor added. 

“Our computer can spot a fake far more accurately than a human,” he said. “Is that a form of artistic appreciation? In a way, artificial intelligence does appreciate a Jackson Pollock painting.”

Story by Nicole Krueger
Photos by Chris Larsen