NVIDIA has announced Canvas, an app that is available as a free beta that it says brings real-time painting tools to anyone with an NVIDIA RTX GPU. It uses artificial intelligence to make rudimentary sketches or doodles into photorealistic scenes.
Canvas is part of NVIDIA Studio, which is a program that provides artists and creators with both hardware and software tools to assist in bringing creative visions to life. Canvas appears to be the consumer-facing version of the app that NVIDIA showcased in 2019, but what is shown here is far more advanced than what was teased back then.
In Canvas, users can start sketching simple shapes and lines using a palette of 15 tools such as mountains, clouds, grass, or weather effects and the artificial intelligence model will turn those into a photorealistic scene in real-time.
What looks like a sketch made in Microsoft Paint is transformed into a lifelike scene instantly and with jaw-dropping quality.
“The tool allows artists to use style filters, changing a generated image to adopt the style of a particular painter,” NVIDIA explains. “NVIDIA Canvas isn’t just stitching together pieces of other images, or cutting and pasting textures, but creating brand new images, just like an artist would.”
The algorithm is aided by the tools palette in Canvas so that it understands what the artist is intending to show, but the results are still extremely impressive. NVIDIA says that the creative applications of the tool are “endless” and that it can help artists ideate faster than before, which is normally a far more time-consuming task.
“It can also bring time savings to a range of creators and designers,” NVIDIA explains. “Consider an art director on a time crunch who needs to refine the hero product in an ad campaign. A landscape will complete the shot. Using NVIDIA Canvas, the director is only a few clicks from generating several beautifully rendered images. In a matter of moments, the tool provides multiple options for the client to review, saving precious time before finalizing the featured product.”
Canvas uses a form of artificial intelligence called “generative adversarial networks,” or GAN. It is the same technology that NVIDIA has used in previous AI algorithms, like its tool that can create photorealistic faces. GANs consist of a generator and a discriminator that work together. A generator aims to convert material maps into images, and the discriminator understands certain aspects of what is generated. For example, the discriminator knows that water has reflections, and can tell the generator to create a representation of that.
“To deliver realistic portraits for artists, the GAN has been trained on an NVIDIA DGX system using over 5 million images,” NVIDIA says.
The Canvas beta can be downloaded for free now, and NVIDIA is soliciting feedback on its forums.
Author: Jaron Schneider
Source: Petapixel
