Retouchers can transform the most mundane of photos into truly stunning ones – but it takes a great deal of skill and time. A joint Adobe-Cornell collaboration has just demonstrated that fully-automated AI systems can do the same thing, suggesting a tool that is likely to make it into a future version of Photoshop or Lightroom at some point.
Adobe says that the software takes a similar approach to the popular Prisma app, which transforms photos into simulated paintings which copy the style of famous artists.
The sample results shown in the research paper look truly astonishing – albeit in low-res form. The tool can convincingly turn day into night, summer into fall, black and white into color.
Our approach builds upon recent work on painterly transfer that separates style from the content of an image by considering different layers of a neural network. However, as is, this approach is not suitable for photorealistic style transfer. Even when both the input and reference images are photographs, the output still exhibits distortions reminiscent of a painting. Our contribution is to constrain the transformation from the input to the output to be locally affine in colorspace, and to express this constraint as a custom CNN layer through which we can backpropagate. We show that this approach successfully suppresses distortion and yields satisfying photorealistic style transfers in a broad variety of scenarios, including transfer of the time of day, weather, season, and artistic edits.
As TNW notes, the way that the app avoids the types of errors typically associated with other automated retouching tools is its real strength.
If I were a photo retoucher, I’d be looking rather nervously over my shoulder right about now.
