Recent projects used machine learning to resurrect paintings by Klimt and Rembrandt. They raise questions about what computers can understand about art.

Full Story:

IN 1945, FIRE claimed three of Gustav Klimt’s most controversial paintings. Commissioned in 1,894 for the University of Vienna, “the Faculty Paintings”—as they became known—were unlike any of the Austrian symbolist’s previous work. As soon as he presented them, critics were in an uproar over their dramatic departure from the aesthetics of the time. Professors at the university rejected them immediately, and Klimt withdrew from the project. Soon thereafter, the works found their way into other collections. During World War II, they were placed in a castle north of Vienna for safekeeping, but the castle burned down, and the paintings presumably went with it. All that remains today are some black-and-white photographs and writings from the time. Yet I am staring right at them.

Well, not the paintings themselves. Franz Smola, a Klimt expert, and Emil Wallner, a machine learning researcher, spent six months combining their expertise to revive Klimt’s lost work. It’s been a laborious process, one that started with those black-and-white photos and then incorporated artificial intelligence and scores of intel about the painter’s art, in an attempt to recreate what those lost paintings might have looked like. The results are what Smola and Wallner are showing me—and even they are taken aback by the captivating technicolor images the AI produced.

Let’s make one thing clear: No one is saying this AI is bringing back Klimt’s original works. “It’s not a process of recreating the actual colors, it is re-colorizing the photographs,” Smola is quick to note. “The medium of photography is already an abstraction from the real works.” What machine learning is doing is providing a glimpse of something that was believed to be lost for decades.