Gearhead: Arena & Gearhead 2

It’s been a while since I talked about the Gearhead mecha roguelike series.

In the meantime, the site has been revised, the source code has been migrated to GitHub, and you can find the games on itch.io. Gearhead has some interesting innovations in the roguelike space: you can pilot giant mecha; there’s an ongoing, generated plot that’s different each game; and it uses randomly-generated conversation responses to fill out its dialogue system.

https://gearhead.itch.io/gearhead-arena

Most interestingly, Joseph Hewett has published some notes on the random story generation in Gearhead. The generator uses a “big list of story fragments” and an algorithm for arranging them. According to him, this approach was inspired by reader response criticism, Scott McCloud’s explanation of closure, and Propp’s narrative functions. Each story chunk (called a “Plot”) is placed in a context of three variables: Enemy, Mystery, and Misfortune. These context variables dictate which Plots can be picked when the algorithm needs a new core story Plot.

Joseph points out that many different methods could be used to arrange these Plot chunks, from simple Markov chains to more complex methods. (You could, for example, use Tracery if you feed the plots into it as strings.)

http://www.gearheadrpg.com/

Eye Gaze Warping

These are still images, plus DeepWarp eye retargeting and some depth trickery.

There’s a lot of things that come to mind every time I see a project like this: The Treachery of Images; the flowering of a new era of computationally-aided art; how image enhancement will soon let us make bigger, sharper neural-network outputs.

Photography isn’t truth: the image is not the thing. From the very beginning, composition and photomontage made photos less-than-totally-reliable, but the degree to which they can be altered today makes chain-of-custody more important than ever.

(For that matter, photorealism is not reality. The images we accept as “real” are different for different cultures and contexts: a film without cuts appears dreamlike and unreal.)

But that’s the very ability that lets us create images that are more-real-than-real, to express things artistically that artists could previously only do with great labor, if at all. While there are still technical and economic barriers to making these tools widely accessible, it’s a lot more accessible than, say, a Renaissance artist who had to employ an entire studio of assistants and cater solely to patrons who could afford it.

And, of course, the blurriness that limits how far this technique can currently be pushed will be alleviated as new deep learning research continues.

http://alteredqualia.com/xg/examples/eyes_gaze2.html

Circuit Trees

One thing I really like is when artists talk about their process: why they made the decisions that they did, what missteps they explored along the way, how they chose what to pursue and what to leave behind. This is especially interesting for generative projects, because it’s the process of making a process. It sometimes feels like shaping a living thing, a digital form of bonsai.

Which is also a literal description of this clever little project by Maciej Konieczny, growing trees out of circuit-board-ish lines. And the write-up is a great read for how one artist approached their generative project.

http://narf.pl/posts/procedural-trees

Deep Image Matting

While I’m talking about neural networks, I should point out this research, which use deep learning to extract better mattes for distinguishing foreground and background objects.

Matting is an old, old problem in visual effects, right back to the very earliest trick pictures. Being able to combine two images into one–or conversely, to remove part of an image to make the combination possible–is fundamental to VFX. Early tricks to achieve it included putting a matte on a glass plane in front of the subject, or blocking off part of the shot so that it wouldn’t be exposed. Techniques such as bluescreen made it easier, but even today separating the foreground and background can be difficult and expensive. Especially for hair.

Existing techniques use color information: bluescreen works by removing all of the blue from an image, so in compositing you just throw away anything that’s too blue. In practice, you have to compensate for color spill, where the background colors bleed onto the actors in the foreground. There’s many ways of dealing with this, but fine details such as hair or fur remain difficult to process.

Enter this technique: instead of using the color information, it trains a neural net to look for the structures that mark the boundaries before foreground and background. Together with a large dataset of example objects, it produces rather effective results, even on images with no clear color separation.

https://arxiv.org/pdf/1703.03872.pdf

What is a random seed?

I’ve gotten questions about how to use random seeds. They come up a lot in discussions of things like Osmos’s fixed level generation, but it can be tricky to jump from knowing they exist to figuring out how to use them.

To understand random seeds, it helps to know how computers generate random numbers. Most random number generators are actually pseudorandom number generators. When computers do an operation, they’re supposed to get the same result every time. That is, they’re deterministic. PRNGs use some clever math to take one number and jump to another number in a way that’s unpredictable.

Thus, the RNGs that we use output a sequence of numbers, each one based on the last one that was generated. A seed is just the starting number.

This can lead to some unexpected technical issues if you try to use a random number generator without considering how they work under the hood. But as a general rule what you want to do is to set the seed at the start of your generation and then use that generator for all of your calculations that need a deterministic, repeatable result.

(I sometimes set up two RNGs: one for the important calculations that other stuff depends on and one for the incidental stuff like randomizing the paths of particles, where no one will notice if they’re not exactly the same.)

Most languages and libraries have their own way of managing randomness. Processing has a randomSeed() function to set the seed, while C# uses the dotNet Random class. Same idea either way.

CycleGAN

One big drawback of previous style-transfer methods was that you needed to train the network on image pairs. In order to figure out the similarities you’d need something like a photo and a painting of a photo. Unfortunately, there aren’t many examples of that in the wild. Things like semantic annotations helped, and there have been attempts with automated processes, but this was a general limitation.

As you might guess, that’s not true anymore. Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros,, the same team who brought us pix2pix have come up with a way to do the training without paired images.

The results are, well, pretty good:

Remember, the one of the limitations of things like edges2cats is that the edges were created with an automated process that missed a lot of details or highlighted irrelevant ones. Being able to use completely separate datasets for the training opens up a host of new options.

https://arxiv.org/abs/1703.10593

https://github.com/junyanz/CycleGAN

Travels in Od

At this point, you can assume that I’ve met a lot of people who are working on generative projects. (Though I’m always looking to meet more!) So I happened to be on the same train where Kate Compton was putting together this generative train ride with Tracery-powered text, though I didn’t see it until much later.

It’s mostly self-playing, though you can stop the train at stations to pick up passengers. Or just watch the generated landscape roll by as you are informed of the local drinking songs or favorite blessings of the lands you travel through.

https://galaxykate.itch.io/travels-in-od

a strange voyage

This is one of my favorite new Twitter bots: @str_voyage, by @joebaxterwebb. It tells the story of a sea-going community, and what they encounter.

What impresses me the most here is the effectiveness of the storytelling. Entries seldom feel robot: the repetition that is there feels like the rhythms of the voyage rather than the tick of the machine. Characters emerge as if from a fog and recede again: elders telling stories, children playing pretend. Strange sailors, deserted towns, lost islands. The cry of a sea-bird.

I think that one reason why it works is the sheer variety of different things to talk about: each message is unique among its neighbors. It also uses cycles and events to give a sense of time, as sunset is followed by the stars emerging, or a storm engulfs the ship. Though the most important aspect may be the framing: a voyage brings a narrative along with it, and even brief entries can be evocative and in-character.

The bot feels like Earthsea, like The Quiet Year, like one of  Marshall Miller’s Dungeon World starters: community, poetic vignettes, and the sea.

https://twitter.com/str_voyage

Osmos

The 2009 puzzle game Osmos has a couple of interesting procedural generation features. First, it has an infinite-level mode where you can continue to play new versions of the levels. But the second part is what makes that possible in the first place: all of the levels were built with procedural generation.

The canonical version of each level is generated from a fixed seed, but it uses the same generators as the “forever” levels.

This is a pretty effective way to get the best of both worlds: a flexible generator that lets the designers rapidly iterate on levels while not caring about the details, while also being able to carefully manage the difficulty progression. And, of course, once you have the working generator, you can just vary the seed to get levels that feel similar but are different enough to be a new-ish puzzle.

It’s also one way around the 10,000 bowls of oatmeal problem: the game comes with 47 flavors of oatmeal, and when players request a new variation they’re explicitly asking for a variation on that particular flavor. They want a semi-predictable outcome. The player’s expectations for the generator line up with what it can produce.

http://www.osmos-game.com/

Practical Procedural Generation for Everyone

One of the talks at GDC that I went to was, of course, Practical Procedural Generation for Everyone by mad scientist Kate Compton.

An extension of the encyclopedia of techniques that she presented in her latest zine, this lightning-fast talk was packed with stuff about all kinds of generative methods. Even if you know a lot about generative things already, you’re likely to encounter something new here, on top of a whole lot of ideas for how to make things better or combine them in new ways.

Best of all, the recording of the talk is available to the public, so you can watch it too:

http://www.gdcvault.com/play/1024213/Practical-Procedural-Generation-for