Spaceship Generator

This spaceship generator by Michael Davies has been popular this week.

It’s written in python and runs in Blender. The method it uses to generate geometry is straightforward but highly effective. Extruding out a basic hull and then decorating it results in convincing-looking ships. 

Michael originally created for the Reddit /r/proceduralgeneration June 2016 monthly challenge but it’s been updated since.

Here’s a video of it in action:

I generated a few ships with it, as you can see. Even the relatively simple geometry and texturing can look pretty convincing with the right lighting.

Procedural generation doesn’t need to be direct geometry manipulation: there and many “higher level” techniques that can be just as effective. And, as you can see, even directly pulling the polygons has its limits. This particular generation approach is pretty easy to adapt and extend.

yhancik:

There’s something slightly sinister about it

(tagging @procedural-generation)

Neural Network Learns to Generate Voice

This is cool, not least of which because it lets you see the process of training a neural network using torch-rnn.

If you’ve never done it before, I recommend training a neural net just so you can watch it learn. (Another reason why it’s important to periodically save the state and settings.) 

I suspect, based on how much data is typically needed for text generation, that a lot more training data is needed to produce words from scratch. And existing open-source speech-synthesis does a decent job if you just need intelligible spoken words. Still, if you wanted to create something that could produce Simlish-like dialog for your next project this would be an interesting way to do it.

Since torch-rnn works on characters without any context, the audio was converted into UTF-8 text, fed into torch-rnn, and then the resulting text converted back into audio. A sound-aware implementation or a better encoding of the audio data would probably improve the performance, but there’s something uncanny about the total contextless learning.

StyLit: illumination in 3D renderings styled from sketches

I’ve been talking about artistic collaboration with the computer a lot. Here’s a perfect example of how that can work.

This research, by Jakub Fišer, Ondřej Jamriška, Michal Lukáč, Jingwan Lu, Paul Asente, Eli Shechtman, and Daniel Sýkora, can use hand-drawn illumination examples to stylize 3D images. 

The idea of using a guide sphere goes back to The Lit Sphere research from 2001, which used normal mapping to apply the sketched texture. Using illumination instead lets the artist have more direct control over the part that really matters: the way that light interacts with the subject. Instead of a direct pixel mapping, it uses a patches to transfer the texture of the drawing.

While there’s been some prior work done with Light Path Expressions, but the really striking thing about this particular example is how interactive it is. As you can see in the video, being able to paint the illumination live and adjust it quickly makes it much easier for the artists to control.

To me, this highlights the importance of flexibility. Generative tools are valuable not just because they can innovate on their own, but also because they can give artists the flexibility to control the parts of the process that matter to them.

(via https://www.youtube.com/watch?v=F6ex4pogilA)

Sugarscape (Epstein & Axtel)

Today, let’s talk about artificial life. Specifically, one of the notable pioneering projects: Sugarscape.

Sugarscape is an agent-based social simulation. An agent is an individual entity that makes decisions. The simulation is the emergent result of a lot of little, simple decisions. Agent-based simulations are relatively common now. Dwarf Fortress is perhaps the most obviously famous example, but a lot of game AI uses agents.

Devised by Joshua M. Epstein and Robert Axtell, Sugarscape agents live in a grid. Some of the cells in the grid have sugar, which the agents eat. Each turn in the simulation, they move around and harvest sugar. Depending on the model being tested, they can also reproduce, deal with sugar depletion, inherit sources of sugar, negotiate with other agents, and so on.

The MASON Multiagent Simulation Toolkit and Ascape both include implementations of Sugarscape, if you wanted to play with it yourself. (That’s where I recorded the animations above.)

Agent-based models didn’t stop with Sugarscape, of course. In the social sciences, one of the famous applications of agent-based models is the Artificial Anasazi model, created to investigate what happened to the native inhabitants of Long House Valley in Arizona, who abandoned the valley around 1300 A.D. Ascape has an implementation of the simulation, which looks like this:

These days, researchers are using agent based models with millions of agents running on GPUs and superclusters.

2001: A Picasso Odyssey

Bhautik Joshi ran 2001 through a style tranfer neural net. It’s the way Kubrick would have wanted it.   

All joking aside, I’d be fascinated to see how some famous directors of the past would have used style transfer. I’d guess that some would be more inventive than others: Hitchcock would use it for a one-off effect, Tarkovsky would work it into everything as long as he could tightly author the result, Welles would incorporate it into a larger illusion. Or painters: Dali would embrace it but be disappointed with the results. And despite how well Van Gogh’s style works for style transfer, I suspect that he would be frustrated if he had to paint them specifically for that purpose. 

Really, I think that Impressionists in general would probably be taken slightly aback at the idea of a machine that can paint Impressionist paintings. The relationship between Impressionism and early photography is complex, but in general the Impressionists were trying to represent what the camera couldn’t capture.

Based on his experiments with monotypes and photography, I suspect that Edgar Degas might have embraced style transfer, though he would probably have made prints and then put pastels on top of them. 

And I’m completely confident that Raphael would have embraced the technology…but would only use it so that his workshop assistants could churn out greeting cards faster.

But they didn’t get the chance. It’s left to the artists of today to find uses for these tools. Like many artistic mediums, cinematography is a language. A language in which the meaning of style transfer hasn’t been found yet. Maybe one of you will be the one to find it. What would you use it to say?

I’m very much not an expert on neural nets, despite playing around with them on occasion, so I asked someone who is. Alex J. Champandard very graciously answered my questions about your question. 

I’ve elaborated on his responses so any mistakes in explaining this are my own. With that in mind, let’s get started:

A neural network is trained on a set of training data. Ideally, you want it to be able to generalize from that set to the larger set that you actually want to classify. If you sample a trained neural network, such an an implementation of char-rnn or the Magic: the Gathering card generators, the stuff it creates should already be mostly unique within that sample.

When a sample repeats itself, that’s due to over-fitting. The neural network has learned the training data too well, and failed to generalize it. The most common reason for this is when the training data is too small–the neural network essentially memorizes the training data by rote. But avoiding overfitting is a frequent training problem. 

And if that wasn’t enough, the training can also fail in the other direction, as Alex explains: “Under-fitting means everything is averaged out / mushy / blurred.”

So a well-trained network, by definition, is much less likely to repeat itself. But what if you still want to make absolutely sure that it doesn’t?

For the purposes of your question we can consider the neural network to be a deterministic function: the same input into the trained neural network will produce the same output. (There are stochastic neural networks where the output is partially random, but for your question we can abstract that by including the random seed.)

To explicitly avoid prior solutions without changing the seed, you’d need to re-train the neural net based on its prior output. You’d end up with a new neural net that’s trained to avoid the things you told it not to repeat. (This is a little bit like adversarial training.)

That’s one way to avoid repeating your output, but it’s not necessarily the best way. When I asked Alex, he said that, "Most neural networks are deterministic so re-training makes sense,“ but that,

"NeuralStyle uses a random seed basically, and a separate optimization algorithm that gives you control outside of the network. There are algorithms in image synthesis that are all about reshuffling image elements, but trying to keep them all in the new image. NeuralStyle does this too in a way, it only can reshuffle things on the page.”

So, based on my current understanding, re-training the network might work fine. But if the original network is generalized enough, it’s probably less efficient than just using a different seed. As with most optimization problems, the best solution depends on exactly what you’re trying to accomplish.

I hope this clarifies your understanding of neural networks. I came across a few other resources while I was researching this, if you’d like to dig further: Awesome Recurrent Neural Networks is a curated list of RNN resources; the M:tG generation community has a tutorial for using the tools they’ve developed to generate new Magic cards; and The Unreasonable Effectiveness of Recurrent Neural Networks is a good introduction to working with recurrent neural networks.

—

I was not previously aware of this phenomenon. I’m not sure if I should thank you for bringing it to my attention.

I have no proof one way or another for how these things are getting made. Though I suspect that cheap labor and stolen content is responsible for quite a bit of it. 

It wouldn’t surprise me if there was a lot of automation involved, but the clips I glanced at were both too varied and too samey to be easily procedurally generated. Parts that would be easy to vary were straight-up copied from video to video, while other parts used radically different art styles. Some of them likely use some of the animation automation tech to puppet the characters, but I’d be a bit surprised if anyone’s set up a wholly automated system.

If there’s a system operating here, I suspect “cottage industry” rather than “rogue AI”. (Of course, there’s a point of view that holds that corporations are a form of AI, but that’s not what you were asking.)

But I wouldn’t be too surprised to find that some of the videos were procedurally generated. It’s not impossible. Millions of the books on Amazon were created automatically, after all. And there are bots that procedurally generate YouTube videos. For that matter, YouTube itself will automatically create videos and channels where it sees a need.

I suspect that one reason why this is a bit mysterious is because it stems from a culture clash between separate audiences on the internet.

YouTube has a thriving, if under-studied, infant audience. I can’t find any information on how widespread the practice is, but there’s some anecdotal evidence that infants and young children are watching a heck of a lot of online video. I ran across some mommy blogs written by parents who were sick of listening to the videos (or their children singing the song).

Four-year-olds don’t chat online much, but they’re perfectly happy watching videos. And, from the content creators’ perspective, toddlers don’t know how to use adblock, and are probably more likely to sit through the ads. There’s a ton of other nursery-rhyme videos on YouTube, profiting off a similar dynamic. Some even offer customization options, creating videos with “your child as the star.”

It does suggest that the future of young children’s entertainment may involve some form of procedural generation. Children are famously willing to watch repetitive content, and may end up being the perfect audience for testing generative algorithms. Some kind of infernal cross between Emily Short’s mushroom principle and Alfred Bester’s earworm from The Demolished Man.

I’m now going to go scrub this song out of my head forever.

Sunspring (2016)

A short film written by a computer.

While most human-computer partnerships have the computer as the junior partner interpreting the human’s requests, this literally flips the script and has the humans interpreting the computer-written screenplay.

Their generator doesn’t have the most sophisticated output I’ve seen. (A Time for Destiny and Terminal Time both have more moment-to-moment coherence, for example.) But those tend to have more human input at the top end, whereas this was written by a neural net that had read a bunch of screenplays.

The human input into Sunspring came at the bottom, as the actors and film crew interpreted the script.

There’s a write-up on Ars Technica with a lot of behind-the-scenes details.

The bot that wrote the script goes by the self-chosen name of Benjamin. It was created by Ross Goodwin. The film was directed by Oscar Sharp and stars Thomas Middleditch, Elisabeth Gray, and Humphrey Ker, with a song by Andrew and Tiger.

Benjamin has a website.

Did I mention that Benjamin chose his own name? They were conducting an interview during the awards ceremony, and the bot told them “My name is Benjamin.” The bot also told them:

What is the future of machine written entertainment?
It’s a bit sudden.
I was thinking of the spirit of the men who found me and the children who were all manipulated and full of children. I was worried about my command. I was the scientist of the Holy Ghost.

(via https://www.youtube.com/watch?v=LY7x2Ihqjmc)

The Mary Jane of Tomorrow

As a prize for IF Comp 2015, Emily Short wrote a short interactive fiction piece called The Mary Jane of Tomorrow as a fanfic for Steph Cherrywell’s Brain Guzzlers from Beyond! In it, you try to train a robot to get a particular set of skills without picking up bad habits (like emo beat poetry). 

There’s a lot of procedurally generated text in it, building on some of the things Emily Short developed for Annals of the Parrigues. Including four different kinds of poetry and a recipe generator.

(There’s some behind-the-scenes spoiler-filled notes on Emily Short’s blog, but you’ll probably want to play the game first.)

Shiv Integer

There’s a long history of art making people upset when it conflicts with their expectations. As it turns out, 3D printer art is no exception.

Thingiverse is a database of models for use in 3D printing. Shiv Integer is a bot-thing by Matthew Plummer-Fernandez and Julien Deswaef that outputs bizarre mashups of CC-licensed 3d Models. It’s generated quite a bit of controversy on Thingiverse. 

Some of its recent remixes include 1541-II Time Project, GY-63 Miniature DVD Cannon, Remote Mario Holder, and Parabolic Pile Treasure.

It reminds me of Le Déjeuner en fourrure and other Surrealist objects. Part of the point is the sheer uselessness of the created objects–objects composed of parts that were often intended as mere utilitarian shims.

Dada and Surrealism have deep links to procedural generation. Dada in particular has come up here before; there’s a lot of interesting links between dadaesque techniques like cut-up and blackout poetry and generative art.