This week at the Ars Electronica festival for art and technology in Linz, Austria, Artnome friend and artist Alexander Reben will be debuting a photo booth that mixes faces together in latent space. Reben calls the project Latent Face, and according to him, “It looks into the stylegan flicker model for the latent vector closest to the input (which is two or more images of faces), then the vectors are operated on to find the midpoint vector and produce the output.” That’s a fancy way of saying it creates something of an average from the two or more faces provided as inputs to produce a convincing-looking digital offspring.

Grant Wood, American Gothic - 1930 — Grant Wood, *American Gothic -* 1930

As an art history nerd, I of course had to ask Alex if it was possible to cross-breed painted portraits. He obliged, and his results are pretty great.

First up, Grant Wood’s American Gothic, maybe the most iconic double portrait of all time. What many people may not know about American Gothic is that the woman in the painting is the artist’s sister, and the man was actually not Wood’s brother-in-law, but his dentist. So this iconic couple were never actually a couple at all, but that did not stop us from wondering what it would look like if they had a child that grew into an adult. You can see the results below and the animation at the top of this post.

Alex Reben, American Gothic Hybrid — Alex Reben, *American Gothic Hybrid*

One of my favorite real-life artist couples is Frida Kahlo and Diego Rivera. It is well-documented in her work and her journals that Frida had always wanted to have a child with her husband Diego. Tragically, Kahlo suffered horrific injuries in a bus accident when she was just 18 that made that difficult.

I asked Alex if we could find the latent vector between the striking self-portraits of Kahlo and Rivera below.

Left: Frida Kahlo, Self-Portrait with Monkey - 1938; Right; Diego Rivera, Self-Portrait - 1949 — Left: Frida Kahlo, *Self-Portrait with Monkey* - 1938; Right; Diego Rivera, *Self-Portrait* - 1949

Alexander Reben, Midpoint between Kahlo and Rivera self-portraits

You can see the animation moving through latent space between the two self-portraits below.

Regular Artnome readers may recall that we averaged every painting by Van Gogh into a single image. For that image, Artnome data scientist Kyle Waters set the aspect ratio for all the portraits to the same dimensions and averaged out the color for each pixel location. It produced the ghostly-looking portrait below.

Kyle Waters, Average van Gogh Portrait — Kyle Waters, *Average van Gogh Portrait*

Unlike Water’s pixel average portrait, Reben’s averages (or more precisely, midpoint images) from StyleGAN are creating entirely new facial structures, not just averages of pixel color values. In this way, it feels more like breeding paintings than creating a visual average. Reben pointed out that, “You can often see this in the hair and facial hair, which many times is a different style than either of the two input images.”

So how does this all work? Okay - let’s start here. Right now you have a digital twin for every photograph ever taken of you in the past, or that will ever be taken of you in the future. These twins exists in a near-infinite matrix of possible faces (called latent space) produced by a deep-learning algorithm developed by Nvidia called StyleGAN.

Like all GANs (generative adversarial networks), StyleGAN is comprised of two neural networks: a generator and a discriminator. In this case, Nvidia trained the generator on 70,000 high-quality images of human faces at 1024×1024 resolution taken from Flickr until it was able to fool the discriminator into believing the images it produced were real human faces. You have probably already seen projects that use this model like This Person Does Not Exist, which became very popular earlier this year.

Reben’s project is interesting to me for two reasons. First, it focuses more on the ability to perform mathematical functions on the faces, and second, it personalizes this technology by finding and cross-breeding images that look like the participants in his photo booth.

In other words, seeing hyper-realistic faces of people that don’t really exist is cool, but discovering I have infinite digital twins that can breed to make offspring with everyone else’s digital twins… well, that is much cooler in my book!

As an example, here is one of my digital twins:

The image on the left is a photo I took with my phone and sent to Alex; the image on the right existed in latent space long before I ever took the photo of myself (in the same way my date of birth existed as a series of numbers before I was born).

To help describe how the process works a bit more, Alex and I took one for the team and made our own handsome man-baby together (whom we call “Jalex”).

Let’s walk through the steps behind the creation of our little bundle of joy.

Step One: Take Two Photographic Portraits

Step Two: Find Our Twins In Latent Space

Reben is using code called StyleGAN Encoder that that identifies and locates the latent vector (the digital twin) within latent space that most resembles the input image. You can actually see it honing in on the right image in latent space in the gifs below.

Step Three: Find the Midpoint

Once Alex located the location of our digital twins in latent space, he could then travel to the arithmetic midpoint between the two and find our “average,” or genetic offspring. I have to say, we produce a pretty handsome looking man-baby.

What Other Math Can We Do?

Again, because our digital twins are described numerically, there are several other mathematical operations that can be performed. If you subtract Alexander’s face from mine, you get this happy little creeper.

And likewise, if you subtract me from Alexander, you get this teen heartthrob.

Kind of weird, right? Whatever features of ours that have been subtracted from the other were apparently pretty important to us looking like our handsome selves.

Reben finds that people respond more to images of people that they know, like the interpolation gif between our two most recent presidents. I find it funny that latent space has no real answer for Trump’s hair.

However, even more engaging than combing pictures of famous people is having your own image combined with someone else. Reben is aware of this, and it is the core idea driving his photo booth at Ars Electronica. For example, I had Reben create a digital offspring for me and my wife. The results were kind of astonishing and bizarre-feeling in a way that you could only feel if it was your image.