Artificial intelligence in graphic editors.

We rarely come across the results of the "creativity" of artificial intelligence in everyday life. But the situation is already changing - AI has already made it to graphics programs and helps to edit or create from scratch landscapes, people's faces and other images.

The Internet stores an incredible amount of images, and they have become an excellent base for training artificial intelligence. Several years ago, neural networks are able to recognize what is depicted in photographs or drawings, and recently they themselves have learned to create graphic objects.

NVIDIA GauGAN Landscape Painter

At the GTC 2019 conference in California, NVIDIA showed the work of its artificial intelligence GauGAN. The AI-program got its name in honor of the French artist Paul Gauguin. GauGAN is a neural network in which one part of it draws an image, and the second evaluates how it needs to be modified to make it even more realistic. To do this, the neural network has analyzed millions of landscapes on the popular photo hosting Flickr and is now able to independently generate landscapes based on sketches. Anyone can sketch a simple drawing with trees, ponds, mountains, clouds, and GauGAN will turn it into a realistic photo with correctly exposed lighting and shadows. If you wish, you can change the season - then frost will appear on the trees instead of green leaves, and the tone of the picture will become colder. The neural network takes into account the smallest details: grass of various shapes and sizes, gradients, textures, insects, even the reflection of objects in water! In general, the artificial intelligence GauGAN is able to paint a picture in digital form, worthy of the brush of a real artist. You can explore some of the capabilities of the GauGAN neural network for free on the official NVIDIA website.

The developers plan that their artificial intelligence-based program will be useful for landscape designers and game developers who make the most realistic images to provide gamers with a deep immersion in the gaming universe.

Google bot that draws cute pictograms

If the previous service is not yet fully available to ordinary users, then everything is easier with a Google bot. You can find and test it at autodraw.com . You can draw any shapes you like - artificial intelligence will analyze them and then correct them. A circle, a flower or a house turned out to be imperfect? No problem: the AI ​​will understand what you are drawing, and will make the lines smoother and clearer, and the object or phenomenon itself will be recognizable. True, so far the bot can handle only the simplest drawings, which can be easily turned into pictograms, and does not understand the text at all.

The yellow circle easily turns into the moon, and for a drawn star the program gives out two dozen options: from an asterisk to an airplane

Microsoft's CariGAN draws cartoons from photographs

Microsoft has been competing with Google for a long time on the field of advanced IT technologies, therefore, both market players have experience in using artificial intelligence in graphics. Microsoft developers have combined two applications, one of which analyzes the portrait, and the second turns it into a cute cartoon. All together is called CariGAN (GAN - Generating Adversarial Network). Jing Liao of the University of Hong Kong and Lu Yuan of Microsoft used a database of several thousand portraits to teach her.

At the first stage, the neural network analyzes the photograph, determines the geometry of the human face and compares it with the cartoon model. In the next step, she applies a special filter to the photo, which turns the portrait into a caricature. Judging by the research of scientists, the neural network in 22.95% of cases coped with drawing cartoons better than a person.

Another interesting feature of CariGANs is that it can analyze videos, extract images of people from there and create cartoons based on them. The AI ​​at the heart of this system can correct cartoons and “try on” various types of characters in a cartoon style.

Image editor based on AI from  MIT

Massachusetts Institute of Technology and IBM have jointly developed an AI-powered graphics application. In  GANPaint Studio, you can add, edit or delete individual objects in photos with one click. It works like this: you load an arbitrary image, and the system will offer several tools for editing it. For example, against the background of the image of the castle, you can draw a tree, on the wall of the house - a door, and inside the room - remove the refrigerator, which interferes with the view of the kitchen. The most interesting thing is that the new object will fit perfectly into the setting, and images with distant objects will look as if they never were there.

Artificial intelligence in GANPaint Studio does not blindly execute user instructions, but does it smartly. If you want to draw a tree or clouds in the middle of the room, he will simply refuse to do so, as such changes will look unrealistic.

Scientists expect GANPaint Studio to help them find fake images on the Internet. But we can also use similar developments for our own benefit. For example, you may not need to clean up things scattered around the apartment before photographing it for a rental site, but simply remove unnecessary items using artificial intelligence.

Adobe helps remove objects from video

Adobe Photoshop users know about a tool called Content-Aware Fill, which removes unwanted objects from photos. The developers of this graphics editor have gone further and implemented this feature in the video editor. In an updated version of After Effects that uses machine learning elements, you can remove objects and people from video clips. This can be done on the Adobe Sensei AI platform. Here, Content-Aware Fill for After Effects first masks the excess object, while Adobe Sensei AI tracks its movement on the remaining frames and carefully removes it.

DeepArts - AI turns photos into a realistic painting

This is the name of a popular online service that brings together dozens of popular art styles. On the site you can get a portrait painted in the style of impressionism or any other of your choice. Artificial intelligence and neural networks are behind this again, and they have learned to make pictures so believable that not everyone can distinguish them from those painted by the artist. The main disadvantage of the service is that each image costs almost two euros, a small image in low resolution is available for review.

Although AI works for us completely free of charge, behind the impressive results of its artistic "creativity" is always the work of many people: scientists, programmers, artists. Who is inventing new areas of application for artificial intelligence and why?

The artificial intelligence painting "Portrait of Edmond de Belamy" was sold at Christie's in London for $ 432,000 - 40 times more expensive than originally expected. In the history of Christie, this is the first piece created not by humans, but by neural networks, behind which is the Obvious art group (France). For their training, a database of 15 thousand paintings, painted over the past four centuries, was used.

What remains behind the scenes

Artificial intelligence can hardly be called a real "author" of the graphic works that it creates. The latter can be decomposed into several components: the output data for analysis (base with images), the learning algorithm, the already trained program, and the immediate result. Both images and programs for their processing are protected by copyright, therefore it cannot be said that the work of AI belongs to AI itself - the owners of such rights were and remain specialists from various fields.

Portrait of Edmond de Belamy

This has changed in recent years, as neural networks often create works using open source software (such as Google TensorFlow or Facebook Torch), but more complex works always have a real author or group of authors. Sometimes neural networks are trained using pictures of the same person. This is what an artist named Anna Ridler does - she uses her own sketches and photographs as a base for AI. She is sure that this is the creative process in which high technologies are involved. But there is an alternative point of view: the algorithm uses someone's images, like any other artist, studies other people's work, sees them in a new way and creates something unique that has artistic value.