December 5, 2020

8 Links

8 Links of Separation

An AI Made For Artists — Create Fantastical Creatures In One Click with Chimera Painter | by Louis (What’s AI) Bouchard | Towards AI | Nov, 2020

This tool is perfect for an artist or a video game designer that has to produce many assets of the same creature and quickly iterate between ideas to find the best creature possible.

Image via Chimera Painter Demo [1]

Well, this can now be done by pressing a single button, and it is already available to try for yourself! This AI uses GANs to create fantastic creatures with multiple styles! Let’s see how it works and how you can use it.

Chimera Painter. Video demo via Chimera Painter Google’s Blog [2]

With Chimera Painter [1] from Google, you only need to draw a creature outline as you can see in these examples, and it will automatically generate a fantastical creature out of it! This outline is only the body parts of the creatures that you need to draw, such as “head” or “wings”, with a specific color and label it using their tool. No need to add any realistic textures or precise details, the AI will do it for you! Then, simply by pressing the “Transform!” button, the model will generate a new creature based on the given outlines. You can always add simple details or body parts and iteratively improve or change the results. Making the tool super simple and useful for any artist that needs to create many assets. You can see this as an assistant made to help you visualize your ideas without spending too much time drawing them.

This AI uses a generative adversarial network architecture. The generative adversarial network, what we call GAN is the base architecture of the network.

GANs are a clever way of training a generative model, which is a model made to generate images of fantastical creatures in this case, by framing the problem as a supervised learning problem with two sub-models. The generator model that we train to generate new examples based on artist-created images and the discriminator model tries to classify examples as either real (from the artist-created training dataset) or fake (created by the generator network).

The two models are trained together in a zero-sum game, in an adversarial way, until the discriminator model is fooled about half the time, meaning the generator model is generating plausible creature examples. In this case, zero-sum means that when the discriminator successfully identifies real and fake samples, it is rewarded, or no change is needed to the model parameters, whereas the generator is penalized with large updates to model parameters. Alternately, when the generator fools the discriminator, it is rewarded, or no change is needed to the model parameters, but the discriminator is penalized and its model parameters are updated.

GAN training process. Image via Packt — Principles of GANs

We can think of the generator as being like a counterfeiter, trying to make fake money, and the discriminator as being like police, trying to allow legitimate money and catch counterfeit money. To succeed in this game, the counterfeiter must learn to make money that is indistinguishable from genuine money, and the generator network must learn to create samples that are drawn from the same distribution as the training data.

They trained multiple generator models using this GAN training method and incorporated the best performing one into Chimera Painter’s demo, which you can try right now, it is the first link in the description. Here, you can see some creatures generated using different models. This best model was decided from artists’ feedback. They were able to generate many different models by varying the hyperparameters of the generator model, which are the parameters that decide the general style of the generated image. This process is called “fine-tuning”, where you iteratively change these parameters to improve your results, using the same architecture and the same training dataset. This means that they can always improve the tool by simply uploading a better model over time, and it will only become better and better.
But this wasn’t a simple task.

Training data example. Image via Chimera Painter Google’s Blog [2]

To create such a realistic and accurate generator model, they needed a huge artist-created dataset with complete segmentations for the tooth, ears, eyes, horns, wings, etc to train it on. In short, each of the 10 000+ training examples in the dataset is composed of two images, one with the fantastical creature results made by an artist, which is the expected output, seen on the left here, and another image with specific colors for each body parts, which we call a “segmentation map” in computer vision, seen on the right, which is our model’s input.

These segmentations are essential for a GAN-trained generator if we want our results to be realistic and if we want it to follow the creature’s proportions, shapes, and textures.

Failed image generation example. Image via Chimera Painter Google’s Blog [2]

Otherwise, it could just merge many body parts and just create something completely weird-looking as you may have seen on the internet somewhere, like this picture mismatching body parts.

The trained model is now available in the Chimera Painter demo [1] which is linked below. I invite you to test it out, it’s really fun to play with! Especially if you are an artist yourself, don’t be afraid of it because it’s a “high tech tool”, it is super easy to use and extremely friendly-user designed. You can easily create many variations of the same creature and visualize the results in a simple click. If you prefer your own tools, they even made it possible to upload your creature outline created in an external program like Photoshop.

Let me know what you think of it!

Here’s a video I made about Chimera as well, with more examples:

Don’t understand how GANs work yet? Here’s a more in-depth explanation: