Interactive 2D Illustrations

I've been wanting to make a game with my wife, Crista Castro, who is an amazing 2D illustrator. My brain went into problem-solving mode. how can I convert her artwork into a functioning interactive scene.

A guess at how a game like Monkey Island 3 could work

The first idea was to recreate the movement from a 2D adventure game: define a walk-able navigation shape, separate background layers so you can walk behind things, fake perspective by scaling the character based on his ground position.

But all of this is really just a way to approximate characters and environments in 3D. The approximation gets more complex and hacky if I want to support things like sloped surfaces or dynamic lights.

So my mind moved on to thinking about the scenes in 3D. This makes the programming very straightforward, but I'd need a process for converting Crista's illustrations into real 3D scenes.

I grabbed an illustration that Crista had done in photoshop, and started exporting layers so each object had a unique texture. I then used a technique for compositing matte paintings called camera projection to project the textures onto rough geometry in Maya. I ran into a ton of issues, but I eventually got the scene rendering in Maya exactly how I wanted.

Crista's original illustration

layers projected onto geometry

Exporting from Maya was another set of headaches. In the end I wrecked the alphas on my textures and got a really buggy scene in Unity.

Creating some basic controls in Unity was a breeze, and I was also able to move the camera around a little to get some nice parallax. I had to be especially careful to not reveal missing chunks of background while moving the camera - something we can compensate for if Crista paints more of the occluded objects in her paintings.

The final interactive scene running in Unity

My biggest take-away: the process of using Maya was painful and tedious, but the results are promising. I think I can write some scripts to automate all of the annoying Maya steps, and I did figure out out (too late) how to fix the busted alpha channels - so I'll keep using this same process, I just need to make it easier on myself.