StyLit: illumination in 3D renderings styled from sketches
I’ve been talking about artistic collaboration with the computer a lot. Here’s a perfect example of how that can work.
This research, by Jakub Fišer, Ondřej Jamriška, Michal Lukáč, Jingwan Lu, Paul Asente, Eli Shechtman, and Daniel Sýkora, can use hand-drawn illumination examples to stylize 3D images.
The idea of using a guide sphere goes back to The Lit Sphere research from 2001, which used normal mapping to apply the sketched texture. Using illumination instead lets the artist have more direct control over the part that really matters: the way that light interacts with the subject. Instead of a direct pixel mapping, it uses a patches to transfer the texture of the drawing.
While there’s been some prior work done with Light Path Expressions, but the really striking thing about this particular example is how interactive it is. As you can see in the video, being able to paint the illumination live and adjust it quickly makes it much easier for the artists to control.
To me, this highlights the importance of flexibility. Generative tools are valuable not just because they can innovate on their own, but also because they can give artists the flexibility to control the parts of the process that matter to them.