neural-doodle: Semantic Style Transfer

I’ve talked about StyleNet and related neural network image generation before, but the level that Alex Champandard has pushed it to is mindblowing. Doodle a simple composition and the algorithm can turn it into a completely different style.

Right now, it’s only been tested on landscapes and portraits, but the potential just in that area is pretty immense. With a public release in the same week week that AlphaGo won the first game in its match against Lee Sedol, computers seem to be finding new ways to replace humans.

So what does this mean for artists? Now that a Photoshop filter to create a Renoir landscape isn’t too far fetched, is there still a place for human artists?

My answer, as always, is that of course there is. Photography didn’t remove human artists. New technology has always been incorporated into painting, from better chemicals, to reference tools, to new ways to think about the world, to reactions that look for the things that the technology can’t do.

But it will change art, especially the commercial production of art and the signaling bound up in fine art painting. Will we view Van Gogh the same way when the Van Gogh filter is all over Instagram? How will this affect the kinds of painting that people are talking about? And, most interesting to me, what new opportunities will this create for artists?

As with every new tool, only some artists will embrace this particular software.

With tools like this, the artist becomes Kasparov’s centaur, a melding of human and machine. Garry Kasparov has also referred to using a computer this way for chess as “Advanced Chess,” and others have called it “Formula 1 for Chess.” You’re still playing the game, but the rote work has been automated, freeing you to find your personal expression.

Using computational tools to make art often feels like the paint guns in Strangethink’s Joy Exhibition. It takes a combination of skill, perseverance, and luck to find the result you envisioned in your head. 

What this definitely won’t do, though, is eliminate the fundamentals. A discipline I learned a long time ago is that sketching out your ideas before you start playing with the machine can help you identify the weaknesses of the software and not be limited by them. It’s sometimes too easy to get stuck on a local peak, because the path lead you there, not realizing that just beyond the fog is a mountain more beautiful and terrifying than you imagined you could imagine. A solid grounding in the basics can help you avoid getting stuck there.

On the flip side, if you’re working with software it’s often useful to learn to speak its language and learn what things it makes easy, and what it could do easily if you just have the right perspective.

I’m looking forward to what happens with neural-doodle in the future. It’s the exact kind of new tool that will inspire that one artist who is dazzled by its possibilities and uses it to make something new that the rest of us have yet to glimpse.

The source code for neural-doodle is on github. You can also read the research paper, with lots more pictures.