Hour-Long Neural Network Video

This project by Damien Henry is an hour-long music video set to Steve Reich that was entirely generated with a neural network. Specifically, a motion-prediction technique, where it tries to predict the future from the current frame..

I admit I’m slightly jealous, since my own neural networks are still training. Though I’m going for a very different approach, and I have to admit that it was pretty ingenious to train the network with what appears to have been footage out of a train window. That gives the training data moment-to-moment consistency but continually changing data in a way that other motion prediction neural networks haven’t tried.

I think we’re rushing through the tech for neural network artistic output: a year ago this would have been impossible, a year from now the cutting edge results will look quite different. There’s already some past results that are visually interesting but you have to dig up the code to recreate them, such as the early stylenet stuff.

You could probably recreate its gradual painting with the current tech, but you’d have to tease out the individual steps.The current stuff is more robust and often better looking, but sometimes there are specific stops along the way that are unique and hard to capture with later versions. (It was also way, way slower, which is one reason why we’ve moved on.)

The specific hallucinatory look of Damien Henry’s train journey is a unique product of this exact point on the neural network development curve. Future neural networks will likely be interesting in their own ways (and faster) but won’t be exactly the same. Together with the sensitivity to the training data, every neural network is unique in its artistic potential.

We can, of course, group them in general categories. And the basic DeepDream puppyslug networks have pretty much mushed together into ten thousand bowls of oatmeal. But there’s still something magical about each moment in this artistic conversation.

https://www.youtube.com/watch?v=-wmtsTuHkt0

(via)